Challenge 5: IoT telemetry ingestion
The situation
Devices in the field send their readings to an ingest service, which stores them in one relational database with a standby. Almost all of the traffic is writes, far more than one leader can take.
Traffic
30,000 requests a second. 1% of requests are for one key at a time.
| Operation | Share | Steps |
|---|---|---|
| Read | 5% | Read data |
| Write | 95% | Write data |
- Read5%Read data
- Write95%Write data
Data
A cache answers 85% of the reads that reach it.
What you are proving
A store that grows with its writes: every reading is kept, spread so that no part of the store, not even the one holding the busiest device, gets more load than it can bear.
Rules
- Readings may be kept in any kind of store.Each reading is written once and never changed, so none needs a transaction.
- A CDN cannot answer these requests.Readings are asked for one device at a time, so no two requests get the same response.
Builds on
Targets
- Serve all 30,000 requests a second
- At least 99.99% available, with no single point of failure
- At most 6,000 USD a month
- Any tier loses one instance: nothing dropped, no tier over 80% busy