Why making a system fast for one user does not make it ready for a million.
What you will learn
Tell a performance problem from a scalability problem.
Weigh scaling up one machine against scaling out to many.
Say what has to change before web servers can scale out.
The fundamental distinction
A system has a performance problem if it is slow for a single user under zero load. Common causes include database queries that use no index, synchronous blocking I/O, heavy cryptographic hashing on the request path, or unoptimized rendering routines.
A system has a scalability problem if it is fast for one user, but degrades or fails as traffic increases. Adding concurrent users causes thread pool exhaustion, database connectionsaturation, lockcontention, or networkbandwidth depletion.
Werner Vogels defines a service as scalable if adding resources increases its performance in proportion to the resources added. Ideally, doubling the servers doubles the throughput:
ΔResourcesΔPerformance≈constantwhat its symbols mean
Vertical scaling (scaling up)
Scaling vertically means adding more CPU cores, RAM, and faster NVMe storage to a single serverinstance.
Advantages: Zero architectural changes; maintains strict ACIDtransactions without distributed consensus; simple single-node debugging.
Disadvantages: Hard hardware ceiling; diminishing returns due to bus and memorycontention; the largest machines cost disproportionately more; one machine remains a single point of failure (SPOF).
Horizontal scaling (scaling out)
Scaling horizontally means adding more discrete compute nodes and distributing requests across them via load balancing. Scaling out on commodity machines is usually more cost-efficient and more available than scaling up one expensive server, but it adds complexity.
Stateless compute: To scale web servers horizontally, session state must be removed from local server memory and kept in a shared store (a database, or a cache such as Redis) or in signed client-side tokens (JWTs).
Downstream load: As the web tier scales out, caches and databases behind it must handle more simultaneous connections.
The load balancer itself: It can become a bottleneck, and a single load balancer is a single point of failure, so production systems run more than one.
Follow a request from the clients: the load balancer sends it to one of four web servers, and that server reads and writes the one database. This is scaling out: to serve more users you add web servers instead of buying a bigger machine, which works only once no session state is kept in a server’s memory. Notice what adding web servers does not multiply: every request still passes the load balancer, and the database carries the load of all the web servers.
Clients: 1 × Users
Load balancer: 2 × Load balancer
Web servers: 4 × App server
Database: 1 × Relational database
Clients sends requests to Load balancer
Load balancer sends requests to Web servers
Web servers sends requests to Database
Key takeaways
Slow for one user at no load is a performance problem; fast for one user but failing as traffic grows is a scalability problem.
A service is scalable when adding resources adds performance in proportion.
Scaling up needs no redesign but hits a hardware ceiling and stays a single point of failure; scaling out is cheaper and more available but adds complexity.
To scale out, keep session state out of servermemory, and expect the load balancer, caches and databases behind the web tier to carry more.