Lesson Beyond 1• 4 min read

Moving bytes

Count megabytes as well as requests, and keep large content off the app servers.

What you will learn

  • Count the megabytes a second a design moves, not only its requests.
  • Tell which tier runs out of network first, and why app servers are the usual one.
  • Keep large content off the app servers: serve it from a CDN and take uploads straight into object storage.

Requests have sizes

Every capacity in this course so far has been a count of requests: an app server serves so many a second, a database leader so many writes. For most requests that is the whole story, because a request and its answer are a few kilobytes and the work on the server costs far more than moving them.

Some content is large. A photo is a few megabytes, a short stretch of video several, a backup or a film gigabytes. Then the same request rate can need very different hardware, because each machine can move only so many bytes a second through its network card. If an app server's network moves about 125 megabytes a second, it can send at most about 30 four-megabyte photos a second, however few requests that is.

So for a system that moves large content, estimate both figures:

load=max⁡(requests per secondrequest capacity, megabytes per secondnetwork capacity)\text{load} = \max\left(\frac{\text{requests per second}}{\text{request capacity}},\ \frac{\text{megabytes per second}}{\text{network capacity}}\right)

and size each tier for the larger of the two.

Find the tier that runs out first

Follow the bytes along each path. A video segment read by a viewer leaves a store, may pass a cache, an app server and a load balancer, and reaches the viewer. Every tier it passes moves those bytes.

The tiers differ by orders of magnitude. A CDN edge is built to send content to users and moves far more than an app server; object storage spreads its traffic over a whole fleet; an app server runs your code and has one ordinary network card. Put a stream of large content through the app servers and they run out of network long before they run out of the work each request needs. You would add servers only to carry bytes they do nothing with.

Keep large content off the app servers

Two techniques keep the bytes away from the tier that is worst at moving them.

  • Reads from a CDN. Popular content is the same for everyone, so a CDN can answer it at an edge near the viewer (lesson 3.2), and fetch the rest from object storage on a miss. Your servers then see only the small requests: a page, a list, a search.
  • Uploads through a signed link. An upload carries the bytes the other way. The app server checks the request and answers with a link to object storage that is valid for a short time and for one object. The client sends the bytes straight to object storage with that link. The app server handles a small request and never touches the content.

Both keep the app tier sized for its real work, the decisions and the small requests, while the bytes take the path built for them.

Viewers anduploaders1× instanceCDN5× instancesLoad balancer2× instancesApp servers6× instancesObject storage1× instanceViewers anduploaders1× instanceCDN5× instancesLoad balancer2× instancesApp servers6× instancesObject storage1× instanceViewers anduploaders1× instanceCDN5× instancesLoad balancer2× instancesApp servers6× instancesObject storage1× instance
Figure 1

Views answered by a CDN, uploads sent through signed links, the app servers left with small requests.

Viewers reach the CDN, which answers the segments at an edge (in this figure, every one of them). Uploads pass the CDN to the app servers, which hand out signed links, so the large bytes go straight to object storage and the app servers see only the small requests. Raise the request rate and watch the CDN fill while the app servers stay lightly loaded: they never carry a video byte.

  • Viewers and uploaders: 1 × Users
  • CDN: 5 × CDN
  • Load balancer: 2 × Load balancer
  • App servers: 6 × App server
  • Object storage: 1 × Object store
  • Viewers and uploaders sends requests to CDN
  • CDN sends requests to Load balancer
  • CDN sends requests to Object storage
  • Load balancer sends requests to App servers
  • App servers sends requests to Object storage

Key takeaways

  • A tier's load is the larger of its request load and its byte load; size it for the larger.
  • App servers usually run out of network first, because they have an ordinary network card and run your code.
  • Serve large reads from a CDN, and send uploads straight to object storage through signed links, so the app servers handle only small requests.
Sources3