Do you struggle around other neurodivergent people?

I have a colleague at work who is autistic and when we interact I find it really difficult. I don’t know if it’s because we both find interaction quite difficult generally but I feel stressed when they’re around me.

I don’t know how it feels for them but when they’re around me I can only describe it as when two magnets repel each other.

I don’t know what it is that makes me struggle to communicate or fills me with dread but it makes me feel like a terrible person. I’ve started to wonder if I dislike being around them because it’s like someone holding up a mirror and showing me a version of myself and reflecting back to me how awkward I must appear to others.

Has anyone else ever had a similar experience?