Are my neighbors keeping my packages?

Hi! So I'm sorry if this is an ignorant post but I'm genuinely curious. My packages always get delivered to the wrong house (only if they're being delivered by Amazon, I've had no problems with AnPost or other delivery carriers) and I've had no issues before.

Initially, my neighbor dropped if off, the next time, I picked it up from their house because we had the same numbered address, mine was just more specific like The Red, or The Blue, so I knew it was delivered there. I did add some specifics to my address in my Amazon account, double checked the pin, even but stil gets delivered in the wrong address.

So black friday comes, and I've ordered more things, not really thinking about this issue. But guess what, they got delivered in the wrong house again. Another package was delivered in the same day but different times (still by Amazon) so I went there that night and asked for my two packages. My neighbor answered and said he met up with the delivery man and realized they had the wrong address and gave the two packages back. I checked the whole of our subdivision and there's no other same number as ours. And the tracking said handed to resident.

So I'm wondering can Amazon take back the packages even though it's not their designated parcel to deliver? Or is my neighbor keeping my packages? And yes, I did have a lengthy conversation with Amazon customer service but that's not the issue. I know I can have it refunded or replaced, still not the issue. So if anyone can answer me, that would be greatly appreciated.