Challenge 12: A ledger of bank transactions
The situation
An account data provider sends each new transaction from users' banks to your API as a webhook, and users open their dashboards to see where their money went. The provider sends a webhook again when it gets no successful answer, so some transactions arrive twice, yet each must be recorded in the ledger once: a repeat would count the same purchase twice in the user's spending. A user opens the same few summaries again and again through the month, so most dashboard reads repeat one already made.
Traffic
3,000 requests a second. 5% of writes repeat an earlier one.
| Operation | Share | Steps |
|---|---|---|
| Read | 30% | Read data |
| Write | 70% | Write data |
- Read30%Read data
- Write70%Write data
Data
A cache answers 85% of the reads that reach it.
What you are proving
Delivery that retries makes repeats normal: whatever records a transaction has to recognize one it has already recorded and leave the ledger as it was, so a repeat costs one lookup and no second row.
Rules
- A CDN does not help here.Each dashboard shows one user's own transactions and budgets, so no copy kept at the edge could answer another user.
- Every transaction is kept in the relational ledger.Spending and budgets are sums over a user's transactions, and the ledger's unique constraint on the provider's transaction ID is what finally turns a repeat away.
Builds on
- 1.2 Response time versus throughput
- 2.2 Availability patterns and failover
- 2.3 Availability in numbers
- 6.1 Caching tiers and cache placement
- 7.1 Queues, task queues and log-based message brokers
- 7.2 Delivery guarantees, ordering and idempotency
- 12.1 Requirements, estimates and the data model
- 12.2 Bank extraction and idempotent ingestion
- 12.4 Categories, budgets and monthly spending
Targets
- Serve all 3,000 requests a second
- At least 99.99% available, with no single point of failure
- At most 2,100 USD a month
- No write stored twice
- Every write kept in a relational database
- Any tier loses one instance: nothing dropped, no tier over 80% busy