Frustration after meet up

To get the normal stuff out of the way... Small 9.5"x6" Weiner and level 9000 professional...

Jokes aside. My wife and I are new to the swinger world in terms of partners but not time. Our first couple has been our only couple for about 5 years. Recently we decided that the other couple wasn't into playing as often as we would like. So we branched out. It took some time but recently found another couple. It went amazing. Had dinner and drinks and went to our house and did the deed. And was in bed by 1am.

Here's were the frustration comes in... Is it supposed to be that easy?!?! The other couple we played with for the longest time wouldn't play until a night out partying at the bar. Sometimes we would go out and not even play because we were all too tired. So it was a huge buildup for nothing. We figured this was a youger person's game and this is just how it is.... But we are pushing 40 with kids so a night out is rare and don't want to waste time. Other couple is the same age and kids but it seemed they were more looking for some bar friends that just happen to be more sexually open if the chance arose.

We have decided not to reach out to the other couple anymore. If they come to us that's fine but no more of this 4am stuff...