Lesson 2.4• 6 min read

Back-of-the-envelope estimation

Powers of two, quick scale heuristics, and latency numbers every engineer must know.

What you will learn

  • Convert between powers of two and powers of ten for storage sizes.
  • Turn daily and monthly volumes into requests per second.
  • Use the latency numbers every programmer should know as orders of magnitude.

The power of quick estimation

In system design interviews and architectural planning, you must quickly verify whether a proposed storage format, network pipe, or database tier will survive real-world scale before writing a line of code.

1. Powers of two and memory capacity

Always remember the 210≈1032^{10} \approx 10^3 rule of thumb:

  • 2102^{10} (1,024 B): ≈1 Thousand (103)→1 KB\approx 1\text{ Thousand } (10^3) \to 1\text{ KB}
  • 2202^{20} (1,048,576 B): ≈1 Million (106)→1 MB\approx 1\text{ Million } (10^6) \to 1\text{ MB}
  • 2302^{30} (1,073,741,824 B): ≈1 Billion (109)→1 GB\approx 1\text{ Billion } (10^9) \to 1\text{ GB}
  • 2402^{40} (1,099,511,627,776 B): ≈1 Trillion (1012)→1 TB\approx 1\text{ Trillion } (10^{12}) \to 1\text{ TB}
  • 2502^{50} (1,125,899,906,842,624 B): ≈1 Quadrillion (1015)→1 PB\approx 1\text{ Quadrillion } (10^{15}) \to 1\text{ PB}

2. Time and throughput heuristics

  • 1 day=24×60×60=86,400 seconds≈105 s1\text{ day} = 24 \times 60 \times 60 = 86,400\text{ seconds} \approx 10^5\text{ s} (easy mental math: divide by 10510^5, then multiply by 1.151.15).
  • 1 million requests / day≈12 RPS1\text{ million requests / day} \approx 12\text{ RPS}
  • 100 million requests / day≈1,200 RPS100\text{ million requests / day} \approx 1,200\text{ RPS}
  • 1 billion requests / day≈12,000 RPS1\text{ billion requests / day} \approx 12,000\text{ RPS}

The primer's worked solutions use a handy conversion for monthly volumes: a month has about 2.52.5 million seconds, so 11 request per second is about 2.52.5 million requests a month, and 400400 requests per second is about 11 billion a month.

3. Latency numbers every programmer should know

These figures were compiled by Jeff Dean and Peter Norvig and date from around 2012. They are in nanoseconds (ns, billionths of a second), microseconds (µs, millionths) and milliseconds (ms, thousandths). Hardware has moved on since, so use them for orders of magnitude, not exact values. They span more than eight orders of magnitude (from 0.5 ns0.5\text{ ns} to 150 ms150\text{ ms} is a factor of 3×1083 \times 10^{8}). If an L1 cache reference were scaled up to 1 second:

  • L1 cache reference (0.5 ns0.5\text{ ns}) →1 second\to 1\text{ second}
  • Branch mispredict (5 ns5\text{ ns}) →10 seconds\to 10\text{ seconds}
  • L2 cache reference (7 ns7\text{ ns}) →14 seconds\to 14\text{ seconds}
  • Mutex lock/unlock (25 ns25\text{ ns}) →50 seconds\to 50\text{ seconds}
  • Main memory reference (100 ns100\text{ ns}) →3.3 minutes\to 3.3\text{ minutes}
  • Read 4 KB randomly from SSD (150 μs150\,\mu\text{s}what its symbols mean) →3.5 days\to 3.5\text{ days}
  • Read 1 MB sequentially from memory (250 μs250\,\mu\text{s}) →5.8 days\to 5.8\text{ days}
  • Round trip within the same datacenter (500 μs500\,\mu\text{s}) →11.6 days\to 11.6\text{ days}
  • Hard disk seek (10 ms10\text{ ms}) →7.6 months\to 7.6\text{ months}
  • Send a packet California →\to Netherlands →\to California (150 ms150\text{ ms}) →9.5 years\to 9.5\text{ years}

Latency numbers, on a log scale

Nanoseconds (ns)Microseconds (µs)Milliseconds (ms)
L1 CPU Cache Reference0.5 ns
Branch mispredict5 ns
L2 CPU Cache Reference7 ns
Mutex lock / unlock25 ns
Main memory (RAM) access100 ns
Read 4 KB Randomly from SSD150 µs
Read 1 MB Sequentially from RAM250 µs
Round trip within a datacenter500 µs (0.5 ms)
Hard disk seek10 ms
Packet round trip, California to the Netherlands150 ms
3 × 10⁸× between an L1 cache reference (0.5 ns) and a transatlantic round trip (150 ms)Source: Jeff Dean and Peter Norvig, ~2012
Client1× instanceCDN1× instanceApp servers4× instancesDatabase1× instanceClient1× instanceCDN1× instanceApp servers4× instancesDatabase1× instanceClient1× instanceCDN1× instanceApp servers4× instancesDatabase1× instance
Figure 4

A request crossing the latency tiers.

Follow a request from the client: it crosses the network to the CDN, goes on to an app server that reads data from memory, and the app server reads the rest from a database on disk. Each step costs a different order of magnitude: a main memory reference about 100 ns, a disk seek about 10 ms, and a round trip across an ocean about 150 ms. A quick estimate adds up these steps to see where the time goes.

  • Client: 1 × Users
  • CDN: 1 × CDN
  • App servers: 4 × App server
  • Database: 1 × Relational database
  • Client sends requests to CDN
  • CDN sends requests to App servers
  • App servers sends requests to Database

Key takeaways

  • 210≈1032^{10} \approx 10^3: KB, MB, GB, TB and PB are each about a thousand times the last.
  • A day is about 10510^5 seconds, so 1 million requests a day is about 12 a second; a month is about 2.5 million seconds, so 1 request a second is about 2.5 million a month.
  • A main-memory reference (100 ns) is about 1,500 times faster than a random 4 KB SSD read (150 µs), and a disk seek (10 ms) is slower still; a round trip across an ocean costs about 150 ms.
Sources5