Challenge 2: An order service that survives failures
The situation
Customers check the status of their orders here. The service runs on one load balancer, a few app servers and one relational database. It is already short of capacity, and losing any one of those machines would stop it or overload the ones left.
Traffic
5,000 requests a second.
| Operation | Share | Steps |
|---|---|---|
| Read | 80% | Read data |
| Write | 20% | Write data |
- Read80%Read data
- Write20%Write data
Data
A cache answers 85% of the reads that reach it.
What you are proving
Redundancy that holds under load: whichever single machine fails, the rest carry every request at the same speed, and the availability you claim follows from the arithmetic.
Rules
- Orders stay in the relational database, and every write is stored in it.The orders are already there.
- A CDN cannot answer these requests.Each request is about one customer's orders, so no two customers get the same response.
- Machines fail independently, one at a time.The availability arithmetic assumes it, as the formulas in lesson 2.3 do.
Builds on
Targets
- Serve all 5,000 requests a second
- At least 99.99% available, with no single point of failure
- At most 1,800 USD a month
- Every write kept in a relational database
- Any tier loses one instance: nothing dropped, no tier over 80% busy