Can you move past a creature you're frightened of?

Say you're adjacent to an enemy who has frightened you. You can't willingly move closer to them, but if you move diagonally left or right and forward so you're alongside them and then keep going then you've never moved closer to them at any point, have you?