A timeline for
a million people

Before any clever optimization, get the two halves clear in your head. A feed system is a feed publishing service and a feed retrieval service sharing a database — and they have opposite shapes.

• Feed publishing (the write path). Someone posts. The system stores the post, then has to make sure it eventually shows up in the timelines of everyone who follows them. The fan-out — how one post reaches many feeds — is the hard part.
• Feed retrieval (the read path). Someone opens the app and wants their timeline: the most recent, most relevant posts from everyone they follow, in order, paginated, fast.

Why insist on separating them? Because every tradeoff in the chapter is really a tradeoff about where the work happens. If you do most of the work on the write path, reads become trivial lookups. If you do it on the read path, writes become trivial inserts. You can't make both free — you're moving the cost around, not deleting it. Naming the two flows up front is what turns a vague "design a feed" into a precise "which path pays?"

The first strategy: the moment someone posts, copy that post into every follower's personal feed — push it into each of their inboxes right then. Each user has a precomputed, ready-to-serve list sitting in a fast store. This is fanout-on-write, also called the push model.

Reads are wonderful. Opening your timeline is a single lookup: "give me my feed list." No gathering, no merging, no sorting at read time — the work was all done when the posts were written. For an app where people read far more than they post (which is almost all of them), pushing the cost onto the rarer write feels like a great trade.

Then it bites. Pushing has three sharp edges:

• Write storms / hotspots. A post by someone with millions of followers becomes millions of writes — one per follower feed. That's a sudden flood concentrated on the write path, and it can hammer specific shards (the celebrity problem, Section 4).
• Wasted work for inactive users. You faithfully write the post into the feeds of people who haven't opened the app in months — and may never. You paid to deliver to an empty room.
• Huge follower counts amplify everything. Write cost grows with follower count, so the users you most want to publish quickly are exactly the ones who are most expensive to fan out.

The opposite strategy: don't precompute anything. When someone posts, you just store the post — one insert, done. The assembly happens at read time: when a user opens their timeline, you look at everyone they follow, pull each of those people's recent posts, merge them together, sort, and return. This is fanout-on-read, the pull model.

Writes are now trivial and cheap — there's no amplification at all, no matter how many followers you have. And there's no wasted work: you never build a feed for someone who isn't looking, because the feed only exists at the moment they ask. Inactive users cost you nothing.

The pain moves to reads. A single timeline request now has to gather posts from potentially thousands of followees, merge and sort them on the spot, and do it inside the few hundred milliseconds a user will tolerate. If you follow a lot of accounts — heavy fanout on the read side — that gather-and-merge gets slow and expensive, and it happens on every open, repeated for every active user.

The crossover point in that widget is the whole insight: push is cheap for normal accounts, pull is cheap for accounts with enormous follower counts. So don't pick one globally — pick per account.

• Normal users → push. They have hundreds or a few thousand followers. Fanning their posts out on write is cheap and gives everyone instant reads.
• Celebrities → pull. An account with tens of millions of followers would cause a write storm on every post. So you don't precompute their posts into feeds at all. Instead, at read time, you check whether the viewer follows any celebrities, pull those posts on the spot, and merge them into the precomputed feed.

So a real timeline is assembled from two sources: the mostly-precomputed push feed (everyone you follow who is "normal") plus a small live pull of the handful of celebrities you follow. Merge by time or rank, return. The expensive write storm never happens, and the read cost stays bounded because nobody follows millions of celebrities — they follow a few.

Where's the switch?

You need a threshold — a follower count above which an account flips from push to pull (often with a buffer zone so accounts don't flap back and forth as their following grows). The exact number is tuned to your fanout capacity, but the principle is fixed: identify the accounts whose fan-out cost would dwarf the read-time merge cost, and exempt them from push.

When you push a post into a million feeds, what exactly do you copy? Not the post. If you copied the full text, image URLs, author profile, and counts into every follower's feed, a single popular post would be duplicated a million times and your cache would explode. Instead, each feed stores just a list of post IDs — tiny integers. The actual post content lives once, in a separate content cache, and you hydrate the IDs into full posts at read time.

So a read is: fetch the viewer's list of post IDs from the feed cache (a fast lookup), then batch-fetch those posts from the content cache (one multi-get), then attach author info. The feed cache stays small because it only ever holds references. This is the single most important space-saving move in the design.

In practice you layer several caches, each with its own job:

• News-feed cache — per user, the ordered list of post IDs that make up their timeline.
• Content cache — the actual posts, keyed by post ID, stored once and shared by every feed that references them.
• Social-graph cache — who follows whom, so the write path can look up "who do I fan out to?" and the read path can look up "whose celebrity posts do I pull?" without hitting the database.
• Action cache — likes, comments, and reshare counts, which change constantly and are kept hot so the hydrated post can show fresh numbers.

The simplest feed is chronological: newest first, pure reverse-time order. It's predictable, easy to merge, and easy to paginate. For a long time it was the default — and some products still offer it as a "latest" toggle.

But most large feeds are ML-ranked: instead of pure time, a model scores each candidate post and sorts by score. The signals usually include:

• Recency — how fresh the post is (old posts decay).
• Affinity — how close the viewer is to the author (you interact a lot → their posts rank higher).
• Engagement — how much the post is being liked, commented on, reshared (popular posts surface).

The score is roughly a weighted blend of these, and the feed is sorted by it. Ranking is why feeds feel "smart" — but it complicates the pure fanout story. If the order depends on signals that change after a post is written (engagement keeps climbing, affinity shifts), you can't just freeze a sorted list into every feed at write time. The precomputed feed becomes a set of candidates, and the final ranking is (re)computed at read time or refreshed periodically. Pure chronological push is simple; ranked feeds push the heavy thinking back toward read time — another reason hybrids and read-time merges dominate real systems.

Underneath the caches sit two durable things: the posts, and the graph of who follows whom.

Posts table

A row per post: post_id (a sortable ID — often time-ordered so "newest first" is built in), user_id of the author, the content (or a pointer to it), and a created_at timestamp. Posts are write-once and read enormously, so they shard cleanly by post_id and cache beautifully.

The follow graph

The relationship "A follows B" is stored so you can answer two questions fast: followers of B (who do I fan out to on write?) and followees of A (whose posts do I gather on read?). You typically store the edge in both directions — denormalized — so each question is a direct lookup instead of a scan.

Sharding the graph is delicate. Shard by user_id and one user's entire follower list lives together — great for fan-out, but a celebrity's list becomes a giant hot partition. You mitigate the same way as everywhere else in this chapter: special-case the huge accounts, and lean on the social-graph cache so the common lookups never touch the cold store.

Denormalize for read speed

The whole feed cache is a denormalization: instead of computing each timeline from posts + graph on every read, you precompute and store the answer (the list of IDs). You trade extra storage and the work of keeping copies in sync for dramatically faster reads — the classic read-optimization bargain.

The push path is the scary one: one popular post can mean millions of feed writes, and you can't do them all synchronously while the poster waits. So fan-out runs as background jobs. The post is stored instantly and a "fan this out" message goes onto a queue; a fleet of workers drains the queue and writes the post ID into follower feeds at its own pace. (That's the queue-and-workers machinery from Chapter 9, applied directly here.)

Because fan-out is asynchronous, the system is eventually consistent: just after a post, some followers already see it while others don't yet — the feed converges over seconds as workers catch up. For a news feed that's perfectly acceptable; nobody needs a post to appear in ten million feeds atomically. The thing to watch is the fan-out backlog: if posts arrive faster than workers can fan them out, the queue grows and the lag stretches. You size workers to the backlog and let the queue absorb spikes.

Two more scaling concerns:

• Pagination with cursors. Feeds are infinite-scroll, so you don't page by offset (which breaks as new posts shift everything down). You page by a cursor — "give me posts older than this ID" — which is stable even as the feed grows on top.
• Hot-key mitigation. A single viral post or a celebrity's feed entry can become a hot key that one cache node can't serve alone. You spread the heat: replicate hot entries across nodes, add a small local cache in front, and (the structural fix) keep celebrities on the pull path so their posts aren't stamped into millions of feeds in the first place.

Here's the heart of both paths in a few lines. The pull version, buildFeed, takes a viewer, the accounts they follow, and the posts, and returns an ordered list of post IDs — it just gathers followees' posts, merges, and sorts. The push version, fanoutOnWrite, takes a new post and the author's followers and returns one entry per follower feed. Notice both are pure: data in, data out, the merge/sort is a plain transformation with no hidden state. Flip the languages — the shape is identical.