The primer’s use cases and estimates: 100 million active users, 6,000 tweets and 100,000 reads a second, 150 TB of new content a month, and the REST API.
What you will learn
State the Twitter timeline and search use cases and assumptions as the primer does.
Estimate tweet storage, fan-out deliveries and read rates.
Size the home timelinecache, including its replicas.
The problem
The primer's Design the Twitter timeline and search notes that designing the Facebook feed and Facebook search are similar questions.
Use cases (from the primer)
A user posts a tweet; the service pushes it to followers, sending push notifications and emails.
A user views the user timeline (their own activity) and the home timeline (activity from people they follow).
A user searches keywords.
The service has high availability.
Out of scope: pushing tweets to Twitter's Firehose and other streams, visibility rules (such as hiding @replies) and analytics.
Assumptions (from the primer)
Traffic is not evenly distributed. Posting should be fast, and so should fanning out to followers, "unless you have millions of followers".
Viewing the timeline and searching should be fast. Twitter is read-heavy, so optimize for fast reads; ingesting tweets is write-heavy.
100 million active users.
500 million tweets per day (15 billion a month), each delivered to 10 followers on average: 5 billion deliveries a day, 150 billion a month.
250 billion read requests and 10 billion searches per month.
Back-of-the-envelope estimates
Using the primer's conversion, 400 requests per second ≈ 1 billion requests per month:
Quantity
Per month
Per second
Read requests
250 billion
250×400=100,000
Tweets
15 billion
15×400=6,000
Fan-out deliveries
150 billion
150×400=60,000
Searches
10 billion
10×400=4,000
Size per tweet (primer): tweet_id 8 bytes, user_id 32 bytes, text 140 bytes, media 10 KB on average. That is about 10 KB per tweet.
10 KB×500×106×30=150 TB of new content per month
150 TB×36=5.4 PB in 3 years
For comparison, Twitter reported 150 million active users and about 5,000 tweets per second on average (7,000 at the daily peak) in 2013, and 300,000 queries per second to generate timelines. So the primer's figures are the right order of magnitude.
Sizing the timeline cache
The primer keeps home timelines in a memorycache such as Redis. Each entry is a tweet ID, a user ID and a metadatabyte: 8+8+1=17 bytes. It keeps "only several hundred tweets for each home timeline", and only for active users. Twitter capped home timelines at 800 entries. With the primer's 100 million active users:
100×106×800×17 B≈1.36 TB
Twitter reported "a couple of terabytes of RAM" for this cluster, and said each tweet is replicated on 3 machines. Replication multiplies the RAM you need, so size the cluster for the copies too.
The user timeline API is similar, and search is GET /api/v1/search?query=hello+world. The primer's request also carries an auth_token, omitted here. For internal calls it suggests RPC.
The primer’s design at its estimated rates: separate paths for posting, reading home timelines and searching, with home timelines in a memory cache.
Follow each use case. Posting goes through the load balancer to the Write API, which stores the tweet in the author’s user timeline in the database and its media in the object store, then hands it to the fan-out service. The fan-out service puts the tweet’s ID into the home timeline of each of the author’s followers in the timeline cache, and passes the tweet to the search cluster to index. Viewing a home timeline goes to the Read API, which reads it from the cache; searching goes to the Search API, which asks the search cluster. The three APIs and the fan-out service are counted at about 1,000 requests a second per server, the lab’s figure, plus one spare. The load balancer is counted at 5,000 a second each and the timeline cache at 12,000 a second per node, for its reads and inserts together, each plus one spare; the database is four shards of a leader and a standby, and the search cluster four shards of five (figure 4).