Lesson Method 4• 5 min read

Step 4: Scale the design

Benchmark, profile, fix the bottleneck the numbers point to, and repeat; every fix is a trade-off.

What you will learn

  • Scale by iterating: benchmark, profile, address the bottleneck, repeat.
  • Let step 1's numbers decide how far scaling needs to go.
  • Say what problem each scaling change solves, and its alternatives.

Step 4: find the bottleneck, then fix it

The last step is to identify and address bottlenecks, given the constraints. The primer lists the usual tools (a load balancer, horizontal scaling, caching, database sharding) and asks whether you need them, not whether you can name them. Discuss the possible solutions and their trade-offs: in the primer's words, everything is a trade-off.

Iterate, do not jump

Seven of the eight worked solutions carry the same warning: do not simply jump from the initial design to the final one. (The AWS solution instead shows the iteration, stage by stage.) Say that you would scale iteratively:

  1. Benchmark or load test.
  2. Profile to find the bottleneck.
  3. Address it, weighing the alternatives and their trade-offs.
  4. Repeat.

For each change, be ready to say what problem it solves. The primer's prompts: what does adding a load balancer with several web servers address? A CDN? Read replicas? What are the alternatives and the trade-offs of each?

Scaling Pastebin from its numbers

Step 1's estimates decide how far step 4 needs to go. In the primer's Pastebin solution:

  • Reads, 40 a second on average and higher at peak: popular pastes are served from a memory cache instead of the database, which also absorbs uneven traffic and spikes. SQL read replicas handle the cache misses, as long as they are not bogged down replicating writes.
  • Writes, 4 a second on average: a single SQL primary with a standby should manage. Only if not would you reach for federation, sharding, denormalization or SQL tuning.
  • Storage, 12.7 GB of new content a month: an object store such as Amazon S3 handles this comfortably.
  • Analytics move to a data warehouse such as Amazon Redshift or Google BigQuery.

Notice what is not in the list: no sharding and no queue. The numbers do not call for them, and adding them anyway would cost complexity for nothing.

Then keep going

With time left, the primer suggests talking points: whether some data belongs in NoSQL, where and when to cache, asynchronism, how services communicate, and security. And scaling is never finished: keep benchmarking and monitoring, and address bottlenecks as they appear.

The lab in this course works the same way. Its calculation panel shows each hop's utilization, so the tier to fix first is the busiest one, and its findings name it.

Client1× instanceLoad balancer2× instancesWrite API2× instancesRead API3× instancesMemory cache2× instancesSQL primary andstandby2× instancesSQL read replicas2× instancesObject store1× instanceClient1× instanceLoad balancer2× instancesWrite API2× instancesRead API3× instancesMemory cache2× instancesSQL primary andstandby2× instancesSQL read replicas2× instancesObject store1× instance
Figure 3

The primer’s scaled Pastebin, simplified: a memory cache and read replicas for reads, one primary for 4 writes a second, and an object store for the text.

Reads and writes now have paths of their own, each sized from step 1’s numbers. The load balancer sends reads to the Read API, which looks first in the memory cache; the misses go to the SQL read replicas, and the text comes from the object store. Writes go to the Write API, which saves the row on the SQL primary, with a standby ready to take over, and the text in the object store. There is no sharding and no queue: 4 writes a second do not call for them. The primer’s scaled design also has DNS, a CDN and an analytics database, which the figure leaves out.

  • Client: 1 × Users
  • Load balancer: 2 × Load balancer
  • Write API: 2 × App server
  • Read API: 3 × App server
  • Memory cache: 2 × Cache
  • SQL primary and standby: 2 × Relational database
  • SQL read replicas: 2 × Read replica
  • Object store: 1 × Object store
  • Client sends requests to Load balancer
  • Load balancer sends requests to Write API
  • Load balancer sends requests to Read API
  • Write API sends requests to SQL primary and standby
  • Write API sends requests to Object store
  • Read API sends requests to Memory cache
  • Memory cache sends requests to SQL read replicas
  • Read API sends requests to Object store

Key takeaways

  • Do not jump from the first sketch to the final design; show each change and the bottleneck it removes.
  • For Pastebin, a cache and read replicas handle reads, one primary with a standby handles 4 writes a second, and an object store handles the content.
  • No sharding and no queue: the numbers do not call for them, and adding them costs complexity for nothing.
Sources4