Is it still considered doing drugs if you just inhale the smoke and smell?

My brother and his friends smoke marijuana and i can always smell it in the house. Is this bad for me?