Messages, instantly,
in order

The web was built for the browser to ask and the server to answer. Chat needs the opposite too: the server must be able to shove a message at you the instant your friend sends it, without you asking. Ch.10 laid out the three ways to fake or fix that — here's the same ladder, viewed through a chat lens:

• Polling — the client asks "anything new?" every few seconds. Simple, but it's mostly wasted questions, and a new message waits up to one polling interval before it shows. Too laggy and too chatty for a chat app.
• Long-polling — the client asks and the server holds the request open until a message arrives (or a timeout). Better latency, but every message still costs a fresh HTTP request, and the server can't easily push to a client mid-request.
• WebSocket — one connection is opened once, upgraded from HTTP, and then stays open in both directions. The client can send and the server can push, at any time, over the same pipe. This is the winner for chat.

A WebSocket starts life as an ordinary HTTP request carrying an Upgrade: websocket header. The server agrees, and from that handshake on the socket is a persistent, full-duplex channel. No new request per message; the server can speak first.

That persistence changes the shape of your backend. Normal API servers are stateless (Ch.1): any request can land on any box, because the box remembers nothing between calls. A chat server is stateful — it is holding Alice's live socket in memory. Alice is now pinned to that specific server for the life of her connection, and the system has to track which server she's on so a message for her can be routed there. Stateful connections are the price of real-time, and most of this chapter is about paying it gracefully.

Don't make one program do everything (we learned that in Ch.1). A chat backend naturally splits into a few specialised services, each with one job:

• Chat servers — hold the live WebSocket connections, receive messages, and push them out. These are the stateful ones.
• Presence servers — track who's online by listening to heartbeats, and tell your friends when your dot turns green or grey.
• API servers — ordinary stateless boxes for everything that isn't real-time: sign-up, login, profile edits, fetching message history, changing settings.
• A key-value store for messages — chat is write-heavy and read-by-recent, so a horizontally scalable KV store (think Cassandra / HBase / a managed equivalent) holds the message history, keyed by conversation.

Now the routing problem from Section 1: when Alice connects, which chat server should she land on, and how does anyone find her later? You can't just pin users randomly and hope. A service-discovery component — classically ZooKeeper — keeps the list of healthy chat servers and helps pick the best one for a new client (by load and location). It also stores the mapping "Alice → chat-server-7" so that a message addressed to Alice can be routed to the exact box holding her socket.

Alice sends "hi" to Bob. Follow it:

1. 1Alice's app sends the message over her open WebSocket to her chat server.
2. 2The chat server asks an ID generator for a fresh message id and stamps the message with it. (More on why this id matters in the next section.)
3. 3The chat server drops the message onto a message sync queue — a per-recipient queue that acts as Bob's inbox.
4. 4If Bob is online, service discovery says which chat server holds his socket; the message is forwarded there and pushed straight down to Bob.
5. 5If Bob is offline, the message simply waits in his sync queue (and triggers a push notification). It's delivered the moment he reconnects.

The message is also written to the KV store so the full history survives. The sync queue is the small, fast, per-user structure that drives live delivery; the KV store is the durable archive.

About that id in step 2: it isn't just a label. A well-chosen message id is what lets every participant agree on the order of a conversation — even though the messages took different network paths and arrived at slightly different times. That's the whole next section.

It's tempting to order messages by their timestamp: stamp each one with the time it was sent and sort. This breaks, quietly and embarrassingly, because clocks disagree. Two chat servers (or two phones) never have exactly the same wall-clock time — they drift by milliseconds or whole seconds. So a message sent after yours can carry an earlier timestamp, and now your conversation renders out of order. This is clock skew, and no amount of NTP fully removes it.

The fix is to stop asking "what time is it?" and start asking "what number is next?" For each conversation (each channel), keep a single counter that hands out strictly increasing ids: 1, 2, 3, 4… Each new message gets the next number. To display a conversation, sort by that number. Because the numbers come from one place per channel and only ever go up, they give a total order that doesn't depend on any clock at all — a monotonic per-channel sequence number.

This also fixes multi-device consistency. Your phone and your laptop both sort by the same sequence numbers, so a conversation looks identical on both, regardless of which device received which message first. Order becomes a property of the data, not of the network's timing.

A 1:1 chat has exactly one recipient. A group has N. When Alice posts to a group, how does it reach everyone? Two strategies, and the right one depends on group size:

• Small groups — copy on send (fanout-on-write). The moment Alice sends, drop a copy of the message into each member's sync queue. Now everyone's inbox is self-contained: delivery is a simple drain of your own queue. Cost: a group of 100 means 100 writes per message. For small groups that's cheap and the read side is dead simple.
• Large groups — one shared queue, pull on read (fanout-on-read). For a group with thousands of members, copying every message into thousands of queues is wasteful. Instead keep one queue per group and have members pull from it, each tracking how far they've read. One write per message, no matter how big the group. Cost: the read side is more work, and you must track each member's read position.

This is the classic write-amplification tradeoff: copy-on-send is cheap to read but expensive to write; shared-pull is cheap to write but more expensive to read. Notice it's the mirror image of the cache/replication tradeoffs from Ch.1 — you're choosing where to pay, on the write path or the read path.

Those grey and blue ticks are a tiny state machine running per message. A message walks through three states:

• Sent — the chat server has accepted it and given it an id. (One tick.)
• Delivered — it reached the recipient's device, which sends back a small ack message. (Two ticks.)
• Read — the recipient actually opened the conversation; the device sends a read-ack. (Blue ticks.)

The mechanism is just more little messages flowing the other way: every real message is answered by an ack that bumps its state, and the sender's UI updates to match. Acks are cheap and they ride the same WebSocket.

In a group, "read" isn't one bit — each member reads at their own pace. So instead of a per-message read flag, keep a per-recipient read pointer: the sequence number of the last message that member has read. "Seen by 4 of 12" is just counting how many members' read pointers have passed this message's sequence number. (There's that per-channel sequence number again, quietly doing more work.)

How does Bob's app know Alice is online? Not by guessing. Alice's app sends a tiny heartbeat ("still here") to a presence server every few seconds. As long as heartbeats keep arriving, Alice is online. If none arrives within a timeout window — say two missed beats — the presence server marks her offline. Heartbeat-with-timeout is far more reliable than trying to catch the moment a connection drops, because crashed clients and dead networks don't send a polite "goodbye."

When Alice's status flips, her friends need to know. The presence server fans out the change — "Alice is online" / "Alice is offline" — to her friends via a pub/sub channel: subscribers to Alice's presence topic get pushed the update over their own WebSockets. This is the same publish/subscribe shape from Ch.10, pointed at status instead of messages.

One real-world wrinkle: connections flap. A phone on a flaky network might drop and reconnect every few seconds, and you do not want Alice's dot strobing green/grey to all her friends. So you debounce: wait a short grace period before broadcasting "offline," and a brief settle before "online," so transient blips never reach anyone's screen.

You close the app on the train. Messages keep arriving. Where do they go? Into your per-recipient sync queue — the same structure from Section 3, now playing the role of an offline inbox. Messages pile up there durably. When you reconnect, your device drains the queue, and a push notification nudges you in the meantime. Nothing is dropped; delivery is simply deferred until you're reachable.

Multi-device is the same idea with one extra number. Each of your devices remembers the highest message id it has already seen — call it cur_max_message_id. On reconnect, a device says "I'm caught up to id 8 040; send me everything after that." The server replays only the newer messages from the sync queue / KV store. Because every device anchors to the same per-channel sequence numbers (Section 4), they all converge to the identical, correctly-ordered conversation — your laptop and your phone agree, no matter which was online when.

And that's the whole system. Here's the arc, one job per layer:

1. 1WebSocket transport — one persistent, bidirectional channel; the server can push.
2. 2Specialised services — stateful chat servers, presence servers, stateless API servers, a KV message store, ZooKeeper to find each other.
3. 31:1 flow — sender → chat server → per-recipient sync queue → recipient (or it waits offline).
4. 4Ordering — a monotonic per-channel sequence number, never wall-clock time.
5. 5Group fanout — copy-on-send for small groups, shared-pull for large ones.
6. 6Receipts — sent/delivered/read via acks; per-member read pointers in groups.
7. 7Presence — heartbeat + timeout, fanned out over pub/sub, debounced against flapping.
8. 8Offline & multi-device — the sync queue is an offline inbox; each device syncs from its cur_max_message_id.

Section 4 claimed a monotonic per-channel sequence number gives a total order independent of any wall clock. Here's the shape in code: a tiny Msg stamped with a seq, an allocator that hands out the next number for a channel, and a buffer that accepts arrivals out of order and keeps them sorted by seq. The wall-clock time rides along as metadata only — it's never the sort key. Flip between the languages; the idea is identical.