Architecture lab

Challenge 14: A distributed cacheTarget 50,000 RPS · p99 ≤ 20 ms

0% busy
App servers (clients)
1x inst50,000 RPS
104% busy
Cache cluster
4x inst50,000 RPS

Design summary

Components

  • App servers (clients): 1 × Users
  • Cache cluster: 4 × Cache, 50,000 requests a second, 104% busy

Connections

  • App servers (clients) sends requests to Cache cluster

48,000 of 50,000 requests a second served, average 16 ms, p99 50 ms, 320 USD a month. Challenge not yet passed: throughput, p99 latency, losing one instance.