do you feel like you have to explain why your baby is small when you tell a stranger their age

anytime we go out i usually wear him and it grabs A LOT of attention. 95% of people say the ‘aww how old’ and when i say he’s almost 3 months i get a look. and it’s not like an awful look but i can tell everyone i say this too kinda have a look like ‘oh he’s small’ or think im lying (which idk why anyone would do that) and then i proceed to say he was born 2 months early. birthday is 8.8 and was suppose to be 9.30. when i say this everyone is like OHH okay that’s why he’s so tiny. people just don’t understand not everyone makes it full term and there are babies who have to come early. idk why i feel like i have to say this every time someone asks. i guess i don’t have to but i just feel like i need to stand up for him in a way? my husband doesn’t understand why i explain that to people and tells me i don’t have to and i guess i don’t but i do? does anyone else feel this way or do this?