Lesson 1.4• 7 min read

Scaling from one user to millions

The systematic step-by-step evolution from a single box to a resilient global architecture.

This walkthrough follows the primer's worked example of growing a service on AWS from one user to tens of millions. Each stage is driven by measuring the bottleneck, not by a user count: the primer gives no fixed thresholds, because they depend entirely on the workload.

What you will learn

  • Follow the primer's six stages of growing a service from one box to tens of millions of users.
  • Name the bottleneck each stage removes.
  • Explain why each stage is triggered by measurement rather than a user count.

Stage 1: A single box

Web server and database run on one machine, with DNS (such as Route 53) mapping the domain to its address. Scale by choosing a bigger box. Simple to run, but there is a hardware ceiling and no redundancy: one crash takes the whole service down.

Stage 2: Separate the database and static content

Move the database to its own managed instance (such as RDS) and static content (files that are the same for every visitor, such as images) to an object store (such as S3). Web and database tiers can now be sized and scaled independently.

Stage 3: Load balancer, redundancy and a CDN

Put a load balancer in front of several stateless web servers spread across availability zones; health checks stop routing to an instance that fails. Run the database as a primary with a standby for failover. Move static (and some dynamic) content to a content delivery network (CDN) such as CloudFront, closer to users.

Stage 4: Caching and read replicas

Web workloads are often read-heavy: the primer's example assumes 100 reads for every write. Add:

  • An in-memory cache (such as ElastiCache running Redis or Memcached) for frequently read data, which cuts both database load and response time.
  • Read replicas that copy the primary asynchronously and serve reads, leaving the primary for writes.

Stage 5: Autoscaling

Instead of sizing for the peak by hand, let the platform add and remove web and application servers as load changes.

Stage 6: Asynchronous work, NoSQL and a warehouse

Move slow work (sending email, processing images, payments) onto a queue (such as SQS): the application server enqueues a job and responds immediately, for example with HTTP 202 Accepted ("accepted for processing, but not yet completed"), and worker services process the queue. Move data that no longer fits the relational model to a NoSQL store, and older data used for analytics to a data warehouse.

Users1× instanceDNS1× instanceCDN1× instanceLoad balancer2× instancesWeb servers8× instancesCache1× instanceQueue1× instanceWorkers2× instancesDatabase2× instancesUsers1× instanceDNS1× instanceCDN1× instanceLoad balancer2× instancesWeb servers8× instancesCache1× instanceQueue1× instanceWorkers2× instancesDatabase2× instances
Figure 4

Every stage of the walkthrough in one design.

Follow a request from the users: DNS turns the name into an address, the CDN answers what it holds, and the rest goes through the load balancer to one of eight stateless web servers. A web server reads from the cache first and from the database on a miss, and puts slow work on a queue so it can answer at once. Each box is a stage of the walkthrough, added when measurement showed the tier before it was the bottleneck.

  • Users: 1 × Users
  • DNS: 1 × DNS
  • CDN: 1 × CDN
  • Load balancer: 2 × Load balancer
  • Web servers: 8 × App server
  • Cache: 1 × Cache
  • Queue: 1 × Queue
  • Workers: 2 × App server
  • Database: 2 × Relational database
  • Users sends requests to DNS
  • DNS sends requests to CDN
  • CDN sends requests to Load balancer
  • Load balancer sends requests to Web servers
  • Web servers sends requests to Cache
  • Web servers sends requests to Queue
  • Web servers sends requests to Database
  • Cache sends requests to Database
  • Queue sends requests to Workers
  • Workers sends requests to Database

Key takeaways

  • Grow by measuring the bottleneck; the primer gives no fixed user-count thresholds because they depend on the workload.
  • First separate the database and static content, then add a load balancer, stateless web servers across zones, a standby database and a CDN.
  • Read-heavy loads get an in-memory cache and read replicas; autoscaling replaces sizing for the peak by hand.
  • Slow work moves onto a queue (answer with 202 Accepted), and data that outgrows the relational model moves to NoSQL or a warehouse.
Sources3