e-Transfer issues and why they happen
UPDATE: We rolled out some quick changes to how we check for e-Transfer status updates, which should hopefully alleviate this issue. Deposit times may be highly variable for the next couple of days.
tl;dr we're waiting on our payment processor to get back to us on an issue with e-transfer deposit recognition.
Technical explanation: our payment processor has an endpoint we have to regularly poll to check on the status of incoming e-Transfer requests (they don't support webhooks). Our task runner checks every 10 minutes, then processes any e-Transfer requests that have changed status since the last check.
When there's a big surge in e-Transfer Requests (like this morning), our calls to that endpoint start timing out because the database lookup time on their end exceeds their HTTP request timeout settings. This endpoint bafflingly does not currently have pagination, which would allow us to grab smaller chunks of data at a time and avoid running into timeout issues.
They've told us that they're rolling out a version with pagination enabled sometime this week, so hopefully this will be an issue of the past. Any interface with the fiat banking system can, unfortunately, be quite brittle.