Can you smell other peoples' homes?

I know, it's a weird one.

My best pal and I have discussed how we are both aware of the distinct smell of any person's house. Every house has its own scent, probably made up of a combo of cooking, people, perfumes, plants, moisture etc. But it's distinct enough that if I catch the scent on my clothes later I can tell whose house I was wearing it in. My boyfriend has no idea what we're talking about.

Is this a totally normal thing that most people sense or is it something super weird that I've always taken for granted?