Challenge 9: Pastebin at scale
The situation
A paste service much busier than the primer's keeps every paste it has stored, and almost every request reads one. Pastes are text only. Most reads are for a small share of popular pastes. Today the text of every paste and the row describing it sit together in one relational database.
Traffic
5,000 requests a second.
| Operation | Share | Steps |
|---|---|---|
| Read | 99% | Read data |
| Write | 1% | Write data |
- Read99%Read data
- Write1%Write data
Data
It keeps 40 TB of content, and 400 GB of metadata about it. A cache answers 85% of the reads that reach it.
What you are proving
Storage chosen for what it holds: the bulky paste text goes where a gigabyte is cheap, and the database keeps only the small rows that a lookup by short link needs.
Rules
- A CDN cannot answer these reads.Every read checks whether its paste has expired, and a copy kept at an edge would go on serving a paste after it expired.
- No paste is deleted to save space.By default a paste never expires, so the stored text only grows.
Builds on
Further reading: System Design Interview, volume 1, chapter 8.
Targets
- Serve all 5,000 requests a second
- At least 99.95% available, with no single point of failure
- At most 3,900 USD a month
- Every write kept in a relational database
- Any tier loses one instance: nothing dropped, no tier over 80% busy