Supply troops to garrison commander still bugged?

I got the quest and had 4 aserai mamluke axemen on hand which I was able to deliver. I obtained 3 more and was told to deliver to Jamayeh Castle. At first I couldn't get in, but after I gifted a bunch of coins to a member of the ruling clan, I was able to enter. But the commander of the garrison won't let me deliver my 3 remaining men (doesn't even acknowledge my quest, even though it's sitting right there in my quest list). Pulling my hair out on this one.