HTTP is a one-way conversation: the browser asks, the server answers, the line goes quiet. This chapter is about the other direction — how a server pushes a new message, a price tick, or a friend's location the instant it happens, from clumsy polling all the way to a full-duplex WebSocket.
The web was built on a simple, one-sided deal: the browser asks a question, the server answers, and then nobody says anything until the browser asks again. That's request/response, and it's beautiful for loading a page. It's terrible for a chat message that needs to appear now.
Plenty of things only feel alive when they update the instant the world changes: a chat thread, a feed, a "typing…" indicator, a sports leaderboard, the dot showing which friends are online, the little markers of people near you on a map. For all of these, the fresh data is born on the server — and the user is staring at a screen that has no idea anything happened.
So the whole chapter is one question asked over and over, with better answers each time: how does the server get a word in? We'll start with the browser nagging on a timer, work up to holding a connection open, then to a one-way server stream, and finally to a permanent two-way pipe — and then deal with the bill that the two-way pipe leaves behind: millions of open connections that have to live somewhere.
Here's the awkward shape of plain HTTP. A connection only carries data after the client asks for it. The server can hold the world's most urgent news — "your friend just messaged you" — and it has no way to tap the browser on the shoulder. It can only wait to be asked, then answer.
For a lot of the web, that's fine. You click a link, you get a page; nothing changes until you click again. But a growing class of products lives or dies on freshness:
The plain-English version of the fix is this: instead of the browser asking "anything new?" over and over, keep one pipe open and let the server speak whenever it wants. Everything in this chapter is a different trade-off on how to do exactly that — how often to ask, how long to wait, and whether the pipe carries traffic in one direction or both.
The dead-simplest answer: have the client ask "anything new?" every few seconds. Set a timer, fire a normal HTTP request, get back either fresh data or an empty "nope," then wait and ask again. This is short polling, and its great virtue is that there's nothing to learn — it's just ordinary requests on a loop, works through every proxy and firewall on earth, and needs zero special server support.
The trouble is it's both wasteful and laggy at the same time, which is an impressive thing to be. Poll every 5 seconds and most of those requests come back empty — pure overhead: a fresh TCP/TLS setup, headers, cookies, an auth check, all to be told "nothing." Yet if a message arrives one second after you polled, the user waits the remaining four seconds to see it. Poll faster and you cut the lag but multiply the wasted requests; poll slower and you save traffic but make the app feel sluggish. There's no setting that's good — only settings that are bad in different ways.
Short polling is the right tool when updates are rare, a few seconds of staleness is fine, and you value simplicity over everything (a dashboard that refreshes every 30 seconds, say). For chat, it's painful.
Long polling fixes the laggy-and-wasteful problem with one clever twist: when the client asks "anything new?", the server doesn't answer right away if there's nothing to say. It just holds the request open — keeps the line hanging — and waits. The moment new data appears, it sends the response immediately. If nothing happens for a while (say 30 seconds), it returns an empty response anyway so the connection doesn't die, and the client instantly opens a new one.
The effect is near-real-time delivery over completely ordinary HTTP. There's no empty "nope" traffic, because the server only replies when it actually has something — and the data arrives within milliseconds of being created, not "whenever the next poll happens to fire." From the client's side it's still just request → response; it simply learned to be patient.
The catch is connection churn. Each delivered message ends the request, so the client must immediately open a fresh one — and a server holding tens of thousands of slow, parked requests ties up resources in a way classic request/response servers weren't designed for. There's also a small gap right after each response, while the client reconnects, where a message could be missed if the server isn't careful to buffer. Long polling was the workhorse of "real-time" web for years, and it's still a solid fallback — but it's a clever hack on top of a protocol that fights it. The next two answers stop fighting and build the open pipe directly.
Watch the requests stack up. Short polling fires constantly and delivers late; long polling fires once per message; SSE and WebSocket open one connection and then just push. The "overhead" stat is the wasted bytes — fresh headers and handshakes — spent per actual event delivered.
What if the client asks once, and the server then keeps the connection open and dribbles messages down it forever? That's Server-Sent Events (SSE). The browser opens a single ordinary HTTP request and the server, instead of answering and hanging up, replies "I'm going to keep talking" and streams a sequence of little text messages down the same pipe as they happen. One request, many responses.
It's a lovely fit for anything that flows from the server to the user: a notification feed, a live activity log, a price ticker, server-pushed progress on a long job. And it comes with two gifts almost for free. First, auto-reconnect: the browser's built-in EventSource client reconnects automatically if the line drops, and can even tell the server the ID of the last message it saw so the server resumes from there. Second, it rides on plain HTTP, so it sails through most proxies and infrastructure without special handling.
The limits are deliberate. SSE is one direction only — server to client. If the user needs to send data back, they do it the normal way, with a separate HTTP request. And the stream is text-only (UTF-8), so binary data has to be encoded. For a huge number of real-time features — "show me new stuff as it appears" — that's exactly enough, and it's far simpler than the alternative.
When both sides need to talk freely — the client sending keystrokes and moves, the server pushing everyone else's — you want a real two-way channel. A WebSocket gives you exactly that: a single long-lived connection that either side can send on, any time, with almost no per-message overhead.
It starts life as a normal HTTP request that politely asks to be upgraded. The client sends a request with an Upgrade: websocket header; the server agrees with a 101 Switching Protocols response; and from that instant the same TCP connection stops speaking HTTP and starts speaking the WebSocket protocol — a thin framing where messages are tiny framed packets rather than full requests with headers. This is the handshake, and after it the connection is full-duplex: both directions, simultaneously, on one pipe.
Because each message is just a small frame (no repeated headers, no cookies, no auth re-check), it's cheap and fast — which is why WebSockets are the default for chat, multiplayer games, collaborative editors, and trading screens. The cost is paid not in bandwidth but in architecture: that connection is open for as long as the user is around, which means the server has to hold it. A request/response server can forget you the instant it replies; a WebSocket server remembers every connected client. That single fact — stateful, long-lived connections — is what the rest of the chapter is about.
Open the connection to watch the upgrade dance, then the steady heartbeat pings keeping it alive. Hit Kill heartbeat and the client stops answering — after a couple of missed beats the server gives up and marks it dead. That's how a server reclaims sockets for clients who vanished without a proper goodbye (closed laptop, dead Wi-Fi).
A million users with chat open means a million open connections, and every one of them is parked on some specific machine — you can't store an open socket in a database. So the first new building block real-time systems need is a tier of connection servers (sometimes called gateway or edge servers) whose entire job is to hold connections and shuttle messages.
Because each connection lives on one machine, you need a directory: which server is user X connected to right now? That mapping — user → connection server — lives in a fast shared registry (Redis is common), so that when a message needs to reach Alice, you can look up "Alice is on server 7" and route it there. As you add or remove servers, a hash ring (consistent hashing, from earlier in the book) helps spread connections evenly and limits how many users get reshuffled.
Two more wrinkles come with long-lived connections. Load balancing is different: a normal load balancer spreads short requests, but here a connection lands once and stays for hours, so you balance the establishment of connections and must avoid dumping everyone onto one box. And you need heartbeats — periodic tiny pings — because TCP won't always tell you a client silently disappeared. If a few beats go unanswered, the server declares the client dead, closes the socket, and cleans up its slot in the registry. Without heartbeats, "ghost" connections pile up and slowly eat the machine.
Now the hard part. A message in a busy group chat must reach 50 people — but those 50 people are scattered across many connection servers. Server A holds twelve of them, server B holds eight, the rest are elsewhere. The connection server that received the message has no direct line to the others, and you absolutely do not want it to know about every server in the fleet.
The answer is a backplane: a shared bus that connection servers publish to and subscribe from. When a message arrives, the receiving server publishes it to a topic (say, the chat-room ID) on the backplane. Every connection server that holds a subscriber to that topic receives it from the backplane and pushes it down the right sockets — and only those. This is publish/subscribe, and the common implementations are Redis pub/sub for low-latency fanout or a message queue (Chapter 9's Kafka/RabbitMQ) when you also want durability and replay.
The beauty is decoupling: the publisher doesn't know or care how many servers or clients exist. It throws the message onto a topic; the backplane handles routing it to whichever servers currently have interested subscribers. Adding capacity is just adding more connection servers that subscribe to the same topics. The publisher's job stays trivially simple no matter how big the fanout grows.
Each client is subscribed to a room (its colored badge). Publish to a room and watch the event flow into the backplane, fan out to only the connection servers holding matching subscribers, and light up only those clients. Servers with no subscriber to that topic are skipped entirely.
There's no single winner; there's a fit for the feature. A quick decision guide:
Then plan for things going wrong, because long-lived connections drop constantly — phones change networks, laptops sleep, corporate proxies kill idle sockets:
Here's the whole chapter as a ladder — each rung adds the server's ability to speak, at the cost of more machinery:
Section 7 claimed the routing core is pure and the sockets live at the edge. Here's the core: a Hub is just a map from topic to the set of clients subscribed to it. subscribe and unsubscribe return a new hub; publish takes a topic and a message and returns the exact list of (clientId, msg) pairs to deliver — pure data in, pure data out. Nothing here opens a socket. The effectful part — actually writing those messages down real connections — wraps this core at the edge. Flip between the languages; the shape is identical.
publish just consults the routing table and returns who-gets-what. The connection servers take that list and do the side effects. Pure routing in the middle, effects only at the edges — that's what makes the fanout logic easy to test and safe to run on every server in the fleet.