Fast travel to character?

On the map when you click on the character I wish you could fast travel to that character. Am I the only one?