System Design real-time delivery
⚡ Part II · Building Blocks · chapter 10 / 24

Push the moment
it changes

HTTP is a one-way conversation: the browser asks, the server answers, the line goes quiet. This chapter is about the other direction — how a server pushes a new message, a price tick, or a friend's location the instant it happens, from clumsy polling all the way to a full-duplex WebSocket.

The web was built on a simple, one-sided deal: the browser asks a question, the server answers, and then nobody says anything until the browser asks again. That's request/response, and it's beautiful for loading a page. It's terrible for a chat message that needs to appear now.

Plenty of things only feel alive when they update the instant the world changes: a chat thread, a feed, a "typing…" indicator, a sports leaderboard, the dot showing which friends are online, the little markers of people near you on a map. For all of these, the fresh data is born on the server — and the user is staring at a screen that has no idea anything happened.

So the whole chapter is one question asked over and over, with better answers each time: how does the server get a word in? We'll start with the browser nagging on a timer, work up to holding a connection open, then to a one-way server stream, and finally to a permanent two-way pipe — and then deal with the bill that the two-way pipe leaves behind: millions of open connections that have to live somewhere.

1The problem: the server can't speak first

Here's the awkward shape of plain HTTP. A connection only carries data after the client asks for it. The server can hold the world's most urgent news — "your friend just messaged you" — and it has no way to tap the browser on the shoulder. It can only wait to be asked, then answer.

For a lot of the web, that's fine. You click a link, you get a page; nothing changes until you click again. But a growing class of products lives or dies on freshness:

  • Chat & messaging — a message typed on one phone must land on another in well under a second.
  • Feeds & notifications — new posts, likes, and alerts should trickle in without a manual refresh.
  • Presence — the green dot that says a contact is online (and goes grey the moment they leave).
  • Leaderboards & live scores — ranks shuffling in real time as results pour in.
  • Nearby friends / live location — moving dots on a map that update as people walk around.

The plain-English version of the fix is this: instead of the browser asking "anything new?" over and over, keep one pipe open and let the server speak whenever it wants. Everything in this chapter is a different trade-off on how to do exactly that — how often to ask, how long to wait, and whether the pipe carries traffic in one direction or both.

💡
The real question isn't "which technology is cool." It's "what's the smallest amount of always-open connection I can get away with for the freshness this feature actually needs?" A stock ticker and a typing indicator deserve different answers.

2Short polling — ask on a timer

The dead-simplest answer: have the client ask "anything new?" every few seconds. Set a timer, fire a normal HTTP request, get back either fresh data or an empty "nope," then wait and ask again. This is short polling, and its great virtue is that there's nothing to learn — it's just ordinary requests on a loop, works through every proxy and firewall on earth, and needs zero special server support.

The trouble is it's both wasteful and laggy at the same time, which is an impressive thing to be. Poll every 5 seconds and most of those requests come back empty — pure overhead: a fresh TCP/TLS setup, headers, cookies, an auth check, all to be told "nothing." Yet if a message arrives one second after you polled, the user waits the remaining four seconds to see it. Poll faster and you cut the lag but multiply the wasted requests; poll slower and you save traffic but make the app feel sluggish. There's no setting that's good — only settings that are bad in different ways.

Short polling is the right tool when updates are rare, a few seconds of staleness is fine, and you value simplicity over everything (a dashboard that refreshes every 30 seconds, say). For chat, it's painful.

⚠️
The polling dilemma: the poll interval is a single dial that trades latency against waste, and you can't win both. Most "empty" responses are cost with no benefit — and at scale, a million idle clients polling every few seconds is a self-inflicted traffic flood.

3Long polling — hold the request open

Long polling fixes the laggy-and-wasteful problem with one clever twist: when the client asks "anything new?", the server doesn't answer right away if there's nothing to say. It just holds the request open — keeps the line hanging — and waits. The moment new data appears, it sends the response immediately. If nothing happens for a while (say 30 seconds), it returns an empty response anyway so the connection doesn't die, and the client instantly opens a new one.

The effect is near-real-time delivery over completely ordinary HTTP. There's no empty "nope" traffic, because the server only replies when it actually has something — and the data arrives within milliseconds of being created, not "whenever the next poll happens to fire." From the client's side it's still just request → response; it simply learned to be patient.

The catch is connection churn. Each delivered message ends the request, so the client must immediately open a fresh one — and a server holding tens of thousands of slow, parked requests ties up resources in a way classic request/response servers weren't designed for. There's also a small gap right after each response, while the client reconnects, where a message could be missed if the server isn't careful to buffer. Long polling was the workhorse of "real-time" web for years, and it's still a solid fallback — but it's a clever hack on top of a protocol that fights it. The next two answers stop fighting and build the open pipe directly.

λ
The framing: short polling samples on a fixed clock; long polling is event-driven — the response is a promise that resolves the instant the server has news. You've turned "ask repeatedly" into "ask once, and I'll get back to you," which is exactly the shape of an async callback.
Interactive · compare the four transports Same event stream, four delivery styles
client → server requests
0
avg latency to deliver
overhead per event

Watch the requests stack up. Short polling fires constantly and delivers late; long polling fires once per message; SSE and WebSocket open one connection and then just push. The "overhead" stat is the wasted bytes — fresh headers and handshakes — spent per actual event delivered.

4Server-Sent Events — a one-way stream

What if the client asks once, and the server then keeps the connection open and dribbles messages down it forever? That's Server-Sent Events (SSE). The browser opens a single ordinary HTTP request and the server, instead of answering and hanging up, replies "I'm going to keep talking" and streams a sequence of little text messages down the same pipe as they happen. One request, many responses.

It's a lovely fit for anything that flows from the server to the user: a notification feed, a live activity log, a price ticker, server-pushed progress on a long job. And it comes with two gifts almost for free. First, auto-reconnect: the browser's built-in EventSource client reconnects automatically if the line drops, and can even tell the server the ID of the last message it saw so the server resumes from there. Second, it rides on plain HTTP, so it sails through most proxies and infrastructure without special handling.

The limits are deliberate. SSE is one direction only — server to client. If the user needs to send data back, they do it the normal way, with a separate HTTP request. And the stream is text-only (UTF-8), so binary data has to be encoded. For a huge number of real-time features — "show me new stuff as it appears" — that's exactly enough, and it's far simpler than the alternative.

📡
SSE is the quiet default that gets skipped too often. If your feature is genuinely one-way (server → client) and text-based, SSE gives you streaming plus free reconnection over boring HTTP — no new protocol, no stateful upgrade dance. Reach for a WebSocket only when you truly need to talk back.

5WebSocket — a permanent two-way pipe

When both sides need to talk freely — the client sending keystrokes and moves, the server pushing everyone else's — you want a real two-way channel. A WebSocket gives you exactly that: a single long-lived connection that either side can send on, any time, with almost no per-message overhead.

It starts life as a normal HTTP request that politely asks to be upgraded. The client sends a request with an Upgrade: websocket header; the server agrees with a 101 Switching Protocols response; and from that instant the same TCP connection stops speaking HTTP and starts speaking the WebSocket protocol — a thin framing where messages are tiny framed packets rather than full requests with headers. This is the handshake, and after it the connection is full-duplex: both directions, simultaneously, on one pipe.

Because each message is just a small frame (no repeated headers, no cookies, no auth re-check), it's cheap and fast — which is why WebSockets are the default for chat, multiplayer games, collaborative editors, and trading screens. The cost is paid not in bandwidth but in architecture: that connection is open for as long as the user is around, which means the server has to hold it. A request/response server can forget you the instant it replies; a WebSocket server remembers every connected client. That single fact — stateful, long-lived connections — is what the rest of the chapter is about.

λ
The framing: request/response is a pure function — request in, response out, no memory. A WebSocket server is the opposite: a long-running process holding a live channel per client. The convenience of pushing freely is bought with state, and state is the thing that's hard to scale.
Interactive · WebSocket handshake + heartbeat Upgrade → 101 → duplex → heartbeats
status: closed

Open the connection to watch the upgrade dance, then the steady heartbeat pings keeping it alive. Hit Kill heartbeat and the client stops answering — after a couple of missed beats the server gives up and marks it dead. That's how a server reclaims sockets for clients who vanished without a proper goodbye (closed laptop, dead Wi-Fi).

6Stateful connection servers — where the sockets live

A million users with chat open means a million open connections, and every one of them is parked on some specific machine — you can't store an open socket in a database. So the first new building block real-time systems need is a tier of connection servers (sometimes called gateway or edge servers) whose entire job is to hold connections and shuttle messages.

Because each connection lives on one machine, you need a directory: which server is user X connected to right now? That mapping — user → connection server — lives in a fast shared registry (Redis is common), so that when a message needs to reach Alice, you can look up "Alice is on server 7" and route it there. As you add or remove servers, a hash ring (consistent hashing, from earlier in the book) helps spread connections evenly and limits how many users get reshuffled.

Two more wrinkles come with long-lived connections. Load balancing is different: a normal load balancer spreads short requests, but here a connection lands once and stays for hours, so you balance the establishment of connections and must avoid dumping everyone onto one box. And you need heartbeats — periodic tiny pings — because TCP won't always tell you a client silently disappeared. If a few beats go unanswered, the server declares the client dead, closes the socket, and cleans up its slot in the registry. Without heartbeats, "ghost" connections pile up and slowly eat the machine.

⚠️
Connections are state, and state is sticky. If a connection server dies, every client on it is disconnected at once and must reconnect (ideally to a different server) — a reconnect storm. Spreading connections, sizing for headroom, and reconnecting with jittered backoff are what keep one server's death from cascading.

7Pub/sub fanout — reaching everyone at once

Now the hard part. A message in a busy group chat must reach 50 people — but those 50 people are scattered across many connection servers. Server A holds twelve of them, server B holds eight, the rest are elsewhere. The connection server that received the message has no direct line to the others, and you absolutely do not want it to know about every server in the fleet.

The answer is a backplane: a shared bus that connection servers publish to and subscribe from. When a message arrives, the receiving server publishes it to a topic (say, the chat-room ID) on the backplane. Every connection server that holds a subscriber to that topic receives it from the backplane and pushes it down the right sockets — and only those. This is publish/subscribe, and the common implementations are Redis pub/sub for low-latency fanout or a message queue (Chapter 9's Kafka/RabbitMQ) when you also want durability and replay.

The beauty is decoupling: the publisher doesn't know or care how many servers or clients exist. It throws the message onto a topic; the backplane handles routing it to whichever servers currently have interested subscribers. Adding capacity is just adding more connection servers that subscribe to the same topics. The publisher's job stays trivially simple no matter how big the fanout grows.

λ
The framing: publish/subscribe is a fan-out over a routing table — one message in, a list of (client, message) pairs out. That routing core is pure data and pure logic; the messy, stateful part (the actual sockets) sits at the edges. That separation is exactly what the code card makes concrete.
Interactive · pub/sub fanout across servers One publish → only the right clients light up
subscribers reached: 0

Each client is subscribed to a room (its colored badge). Publish to a room and watch the event flow into the backplane, fan out to only the connection servers holding matching subscribers, and light up only those clients. Servers with no subscriber to that topic are skipped entirely.

8Choosing well — and failing gracefully

There's no single winner; there's a fit for the feature. A quick decision guide:

  • One-way, server → client, text? Use SSE. Streaming plus free reconnect over plain HTTP — the simplest thing that works for feeds and notifications.
  • Two-way, low-latency, chatty? Use WebSocket. Chat, games, collaboration — accept the stateful-server cost.
  • Rare updates, simplicity above all, or hostile proxies? Polling (long polling for near-real-time, short polling for the truly lazy case) still earns its keep.
  • Worried about scale of open connections? Every persistent transport (SSE, WebSocket) needs the connection-server + backplane machinery from sections 6–7. Polling doesn't — that's its one structural advantage.

Then plan for things going wrong, because long-lived connections drop constantly — phones change networks, laptops sleep, corporate proxies kill idle sockets:

  • Reconnection with jittered exponential backoff, so a server hiccup doesn't trigger every client to reconnect in the same instant (a thundering herd). On reconnect, send the last-seen message ID so the server can replay what was missed.
  • Backpressure — a slow client can't keep up with a fast stream, so its send buffer grows. You must bound it: drop, coalesce, or disconnect, or one slow consumer leaks memory on your server.
  • Graceful degradation — if the WebSocket can't be established (a proxy blocks the upgrade), fall back to long polling. Mature real-time libraries do this automatically; the user just sees a slightly less snappy app instead of a broken one.

Here's the whole chapter as a ladder — each rung adds the server's ability to speak, at the cost of more machinery:

  1. 1Short polling — ask on a timer. Simplest; wasteful and laggy.
  2. 2Long polling — hold the request open until there's data. Near-real-time over plain HTTP; connection churn.
  3. 3SSE — one open HTTP stream, server → client only. Auto-reconnect, text-only.
  4. 4WebSocket — one persistent full-duplex pipe after an upgrade. Bidirectional; stateful servers.
  5. 5Connection servers — hold the sockets; map user → server; heartbeat the dead away.
  6. 6Pub/sub backplane — fan one event out to the servers holding the right subscribers.
  7. 7Plan failure — reconnect with backoff, bound backpressure, degrade to polling.

9A pub/sub hub, as pure routing

Section 7 claimed the routing core is pure and the sockets live at the edge. Here's the core: a Hub is just a map from topic to the set of clients subscribed to it. subscribe and unsubscribe return a new hub; publish takes a topic and a message and returns the exact list of (clientId, msg) pairs to deliver — pure data in, pure data out. Nothing here opens a socket. The effectful part — actually writing those messages down real connections — wraps this core at the edge. Flip between the languages; the shape is identical.

λ
Notice what's not here: no sockets, no IO, no mutation in the core. publish just consults the routing table and returns who-gets-what. The connection servers take that list and do the side effects. Pure routing in the middle, effects only at the edges — that's what makes the fanout logic easy to test and safe to run on every server in the fleet.