My family doesn't know that I'm gay, boyfriend got uncertain

So my boyfriend (37M) and I (28M) got together about a year ago.
I told him at the beginning of our relationship that my family doesn't know that I'm gay, and although I want to tell them eventually, I don't think it will be anytime soon.
For context, my family is very conservative, and I know how would they react. I love them, and I don't want to lose them.
I moved out when I was 19, and they live on the other side of the country, so we don't see each other often. We usually text a few times a week, so I wouldn't say that we're that close, but still I think if I told them the truth, they would break, especially my dad.
Anyway, so I told my boyfriend this when we got together, and he was okay with it.
This week I went to see my parents for the holidays, and I sent a photo that we took at the Christmas tree of me and my family to my boyfriend. He replied with a thumbs up emoji, so I knew that something was wrong.
I asked him, and he said everything is fine at first, then later he told me that he can't believe that I don't know what his problem was.
Then after a few hours he sent a long text message saying that he's talks about me all the time to his parents, because I'm a part of his life, and that it bothers him so much that don't do the same.

I totally understand his feelings, but I think I was clear about this at the beginning.

All of my friends and coworkers know about him, so it's not like I'm hiding him, it's just that I don't feel ready telling it to my parents. I always daydream about telling them, but I'm not strong enough to deal with the consequences.