When a customer pays online, what actually happens in the background? Today you learn the full journey of a transaction — from the moment someone taps "Pay" to the moment money moves.
Think about ordering food at a restaurant.
You tell the waiter what you want → (API) — the waiter takes your order and passes it to the kitchen.
The kitchen has a ticket board with all orders queued up → (Message Queue) — orders are handled one by one in order, so nothing gets lost or mixed up.
The chef cooks it and the order is marked as done → (Database) — the final record is written: "Table 4 had a burger, paid, completed."
Every payment transaction works exactly like this. API takes the request. Queue lines it up safely. Database stores the final result.
An API is basically a gate or a door that your system opens to the outside world. When a customer clicks "Pay Now", their app sends a message to your company's API. The API receives that message, checks if it looks correct, and then passes it inside the system.
Think of it as a receptionist at a bank. You walk in (you = the customer's request), the receptionist checks your ID and your request, and then sends you to the right department. The receptionist doesn't do the actual work — they just receive and route.
In fintech, the API checks things like: Is this a valid request? Does this account exist? Is the amount correct? If yes — it accepts it and moves it along. If no — it immediately sends back an error.
After the API accepts the request, it doesn't process the payment directly. It drops the request into a Message Queue — basically a safe waiting line. The queue holds the transaction until the system is ready to process it.
Why does this exist? Imagine 10,000 people paying at the same time. If all 10,000 hit the database directly at once, it would crash. The queue controls the flow — it feeds the database one by one at a manageable speed. Nothing gets lost even if the system is temporarily busy.
Think of it like a post office sorting room. All the letters (transactions) come in at once, get sorted into a line, and are delivered one by one in the correct order.
The database is where the transaction is finally written and stored forever. Once the message comes out of the queue and gets processed, the result — success or failure — is saved in the database. This becomes the permanent record.
Think of it as the official register at a bank. Every transaction, every balance change, every account update — written in the book permanently. If someone asks "did this payment go through?", you check the database. It's the single source of truth.
As an L2 engineer, you'll often query the database to check if a transaction was recorded, what status it has, and exactly when it happened. This is how you confirm whether a payment truly went through or got stuck somewhere before reaching the DB.
When a client says "a payment failed" or "I don't know if the money went through" — your job is to trace that transaction. That means following it through each layer: API → Queue → Database, and finding exactly where it got stuck or failed.
You do this by checking logs (records of everything the system did) and querying the database (looking up the transaction by its ID or reference number).
| Step | Layer | What You Check | What You Found | Status |
|---|---|---|---|---|
| 01 | API | Did the API receive the request? | API log shows request received at 14:32:03. Returned 200 OK. | ✓ OK |
| 02 | Queue | Did it enter the Message Queue? | Queue log shows TXN-9823 entered at 14:32:03. Picked up by worker at 14:32:04. | ✓ OK |
| 03 | Worker | Did the worker process it? | Worker log shows processing started. Then: ERROR: Bank gateway timeout after 30s |
✗ FAILED |
| 04 | Database | Was anything written to the DB? | DB query shows TXN-9823 exists with Status: FAILED. No money was deducted. | ✗ FAILED |
| 05 | Notification | Was customer notified? | SMS sent: "Your payment of 500 could not be processed. Please try again." | ✓ Sent |
The transaction made it all the way through the API and queue successfully. It failed at the Worker level — specifically when trying to connect to the bank's gateway. The bank's system timed out and didn't respond in time. This is not your system's fault — it's the bank's gateway.
Your next step: raise this with the bank's technical team, share the TXN ID and timestamp, and ask them to check their gateway logs. Update the Jira ticket with your findings.
Client says: "Customer paid but money didn't reach us." You check the DB — transaction shows SUCCESS. Balance updated. This means the money did move. The issue is likely a reporting or notification problem, not a payment problem. Completely different investigation.
Client says: "Payments are very slow today." You check the queue — it has 8,000 messages waiting and only 2 workers processing. The queue is backed up. Solution: scale up the number of workers or find out why they slowed down.
Client says: "Payment failed but customer's account was debited." This is the worst scenario. Money left the customer but didn't arrive. You trace both sides — check your DB and ask the bank. This is called a reconciliation mismatch and gets escalated to L3 immediately.
API is returning 500 errors for all payment requests. No need to trace further — the issue is at the very first step. Check the API server logs, check if the server is overloaded or if a recent deployment broke something.