I think we're all being a little unfair.
Whilst there are lots of thoughtless , rude people out there Im sure that many people say things like ' oh at least you are at home' in a well-meaning way - kind of a 'keep positive / chin up' kind of statement.
Now the ones who say I have a little sideline.......aaaargh!