Getting freaky breaks your oath??

So I'm in my first playthrough right now, just got the game last week, and decided to play a Paladin. I thought being bound by an oath to be a decent person was a cool concept, and since I always try to be a good person in roleplaying games I figured it wouldn't be an issue.

That was until Gale decided to confess his love to me. I told him I loved him back, we got freaky underneath the stars, but when I woke up my character said he felt like he lost something. I didn't realize at the time what happened, so I just kept playing like normal for another hour before getting into a fight and needing to use my Channel Oath. I tried to use it, but it told me I was out of charges. That was weird since I hadn't used it since my last long rest, so I opened my character sheet and saw that it said my oath was broken.

I was confused, thinking maybe I accidentally hit an innocent civilian with a rogue Eldritch Blast, so I started reloading saves. I reloaded each save one by one and it kept saying my oath was broken... Until I got to the save right before Gale confessed. I thought surely there's no way that not rejecting Gale broke my oath, but sure enough when I went through the cutscene again I woke up with my oath broken.

Is there anything I can do to fix this? I want to romance Gale still, but I don't want to suffer never being able to use my Healing Radiance again :(

Is this just a lose/lose situation?

Edit: I did another long rest and the Oathbreaker Knight appeared in my camp, but I never saw a cutscene like I was apparently supposed to. He just appeared here. It seems like I can get my oath back here, but I'm still mad that I have to pay 1k gold and beg for forgiveness for having sex! That's not fair!