Algorithms to Live By · ch.10 · networking
🌐 Chapter 10 · How we connect

Networking:
the protocols of talking

Every conversation, human or machine, has to solve the same three problems — did you hear me, how fast should I talk, and what do we do when we both speak at once. The internet's answers turn out to be surprisingly good advice for people.

Every chapter so far has been about one mind deciding alone. This one is about two of them trying to reach each other across a channel that loses things, delays things, and is shared with everybody else. Which is to say: it is about every conversation you have ever had.

The engineers who built the internet had to answer questions they didn't think of as emotional: how do I know you're listening, how hard should I push, when should I give up, what do I do when I'm overwhelmed. They got answers. The answers are better than ours.

1Two ways to have a conversation

A phone call — the old kind, with copper — reserves a whole line just for you. From the moment you're connected to the moment you hang up, that line is yours. Nobody else can use it. Not even during the silences, and a phone call is mostly silences. You pay for the whole pipe and you use a fraction of it, and in exchange you get something lovely: once you're in, you're in. The line will not degrade because someone else picked up a phone in Ohio.

A letter works nothing like that. You drop it in a box and it takes its chances with every other letter in the country. It gets sorted, trucked, re-sorted, and it finds its own way. Nobody reserved anything for you. Mostly it arrives. Sometimes it doesn't, and nobody tells you.

The internet went with letters. Your video call isn't a line — it's a torrent of tiny envelopes, each addressed and released into the wild, each free to take a different route, arriving out of order and getting reassembled at the far end. The formal names are circuit switching (the phone line) and packet switching (the letters).

The trade is exact and worth saying plainly. Circuits guarantee everything and waste most of it — the line sits idle while you think. Packets waste nothing and guarantee nothing — every scrap of capacity gets used by somebody, and in return nobody is promised anything at all. Packets waste nothing and guarantee nothing. Every remaining idea in this chapter exists to paper over that second half.

📮
This is why network engineers talk about "best effort." The internet does not promise to deliver your packet. It promises to try, and to not lie to you about having tried. Everything you think of as a reliable connection is a fiction assembled on top of that shrug — by counting, checking, and asking again.

2“Are you there?”

You can't just start talking. Before any content moves, both sides need to establish that someone is actually on the other end — and, more subtly, that the other side knows you know. Watch the opening of any phone call and you'll see it happen in about two seconds:

  • “Hello?”  — I'm here. Are you?
  • “Hi, it's me!”  — I hear you, and I'm here too.
  • “Hey!”  — And I hear that you hear me.

Three steps, not two — and the third one isn't politeness padding. After step two, the caller knows the link works in both directions, but the receiver only knows their own words went out; they haven't yet heard anything back confirming it. Step three is what makes the knowledge symmetric. Machines do the identical dance under the name the TCP three-way handshake: SYN, SYN-ACK, ACK. Same three beats, same reason.

Now the part that should unsettle you. Try to extend the pattern to certainty. Two generals are camped on opposite hills with an enemy in the valley between them. They'll win if they attack together and lose if either attacks alone. The only way to coordinate is a messenger who might get caught.

General A sends “attack at dawn.” But she can't attack — she doesn't know it arrived. So B sends back “confirmed.” But B can't attack either — he doesn't know his confirmation arrived, and A won't move without it. So A sends “confirming your confirmation.” And now A can't move, because she doesn't know that got through. You can see where this goes. Every acknowledgement needs an acknowledgement. The regress never bottoms out. This is the two generals problem, and it isn't an engineering gap waiting for a cleverer protocol — it's a proof. Perfect common knowledge over an unreliable channel is impossible. Not hard. Impossible.

So what does TCP do? It stops at three and attacks at dawn. It doesn't achieve certainty; it achieves enough, and then it gets on with it. That's the whole trick, and it is worth sitting with: certainty was never on the menu. Every system that works — every protocol, every marriage, every deal closed over a handshake — has quietly accepted a probability and called it good.

⚔️
The next time you re-read a message before sending it, or want one more confirmation before you commit, notice what you're reaching for. It's the fourth ack. It doesn't exist. The choice was never certainty vs. doubt — it was which doubt you can live with.

3What if it gets lost?

Here is the entire reliability trick, and it fits in one line: number the things you send, say what you got, resend what went missing. That's it. That's how a channel that drops packets at random carries your bank transfer.

The elegant bit is that nobody ever has to report a failure. Suppose I send you packets 1, 2, 3, 4, 5 and you acknowledge 1, 2, 4, 5. I never asked “did 3 arrive?” and you never said “3 is missing.” The hole in the acks told me. Loss is detected by the shape of the silence, not by anyone announcing it. Which means the sender doesn't need a functioning error channel to notice errors — it just needs to be counting.

The human version isn't an analogy so much as the same mechanism running on different hardware. “Mm-hm.” “Right.” “Yeah.” “Go on.” Those aren't filler and they aren't agreement — linguists call them backchannels, and they are acknowledgements. They carry almost no content and they are load-bearing. Their job is to close the loop so the speaker can keep sending.

Which explains a specific modern misery. A video call strips the nodding out — the lag delays it, the tiny window hides it, the muted grid deletes it entirely. You are transmitting into a channel with no acks, so some part of your brain does exactly what TCP does when the acks stop: it assumes loss, slows down, and retransmits. Hence the over-explaining, the “does that make sense?”, the exhaustion. You weren't talking. You were retransmitting into silence for forty minutes.

📡
The uncomfortable corollary: if you go quiet on someone, you haven't given them nothing. You've given them a gap in the acks — and a gap in the acks is not a neutral event. It's the signal for loss. They will assume the message failed and act accordingly: resend, back off, or eventually close the connection.

4How fast should I ask again?

Your message didn't get through. What now? There are two obvious answers and both are wrong. Retry immediately, forever, and you're a denial-of-service attack with good intentions — worse, if the failure was a collision with someone else's message, you'll collide with them again, at exactly the same instant, forever. Give up after one try and you've thrown away a connection that was probably just unlucky.

The answer that works is almost silly: wait twice as long each time. Fail once, wait a second. Fail again, wait two. Then four, eight, sixteen. Keep trying forever, but let the interval grow.

Look at what that single rule buys you. A brief hiccup gets retried almost instantly — you're back within a second, fully responsive. A server that's genuinely down gets gently, gracefully abandoned: after ten failures you're knocking once every seventeen minutes, which is basically nothing, but it's not zero. You never sent a rejection. You never closed the door. If it ever comes back, you'll find it. Your maximum rate is bounded, your total effort is bounded, and yet you never formally gave up. The name is exponential backoff, and it dates to Aloha net — Norm Abramson's 1970s radio network in Hawaii, where every island was shouting on the same frequency and nobody could hear a collision happen.

The uses are the good part:

  • Network retries. The original. Every HTTP client you've ever used does this, or should.
  • Login attempts. The right way to handle a wrong password. Lock the account after three tries and you've handed any stranger a button that disables anyone's account. Instead: after each miss, double the wait before the next attempt. A real user typo-ing twice barely notices. A script trying a million passwords is dead within twenty guesses — not blocked, just slowed to a crawl it can never escape. Nobody gets locked out and nobody gets in.
  • Second chances. This is the humane one. Someone keeps letting you down. The culture offers you two settings: forgive instantly and get hurt again, or cut them off forever. Backoff offers a third. You don't have to revoke anything. You just make the interval longer. Once a week becomes once a month becomes once a year — and each of those is still a real invitation. The relationship is never terminated; its cost simply decays toward nothing. Persistence with a shrinking price.
Interactive · fail it and watch the gap double Same wall clock, two policies
Same rule, three faces
Nothing has failed yet.
Attempts made
0
Next retry in
Total wait
0s
Fixed lane would've tried
0
Hit “It fails again” a few times. Both lanes cover the same stretch of time — watch one of them fade to a polite background hum while the other keeps hammering at exactly the same rate it started with, forever.
🎲
The jitter toggle matters more than it looks. Turn it off and every client that failed at the same moment retries at the same moment — a thousand machines backing off in perfect lockstep, re-colliding in perfect lockstep, doubling in perfect lockstep. That's the thundering herd. The cure is to wait a random amount up to the doubled window. Same growth curve, no synchronised stampede.

5The sawtooth you live in

Backoff told you what to do when a message fails. Now the harder question: how fast should you be sending in the first place? Nobody will tell you the answer. There is no field in any packet that says “the link between you and Frankfurt can take 40 megabits today, and 12 after 8pm.” The capacity is unknown, it's shared, and it changes minute to minute. And yet somehow your connection finds it.

Here's how, and it's two rules: every time something works, speed up a little. The moment something fails, cut your rate in half. That's the whole algorithm. It has a name that's just a description of itself — additive increase, multiplicative decrease, or AIMD — and it is, in a real sense, why the internet works.

Notice how lopsided it is. Success earns you a crumb; failure costs you half your kingdom. That asymmetry is not timidity, it's the load-bearing wall. Every connection on earth is constantly, gently probing upward for a little more, and flinching hard the instant it finds the edge. The result is that nobody ever needs to be told the speed limit. They discover it by walking into it, backing off, and creeping up again — forever. Your connection has never once known the right rate. It has only ever been slightly too slow or slightly too fast, oscillating around a number it will never actually sit on.

And then the genuinely surprising part, which the widget below will show you better than a paragraph can. Put two connections on the same link. They can't see each other. They don't negotiate. They have no idea how many others are out there. Both halve when the link gets full; both creep up at the same crumb rate. Halving shrinks the gap between them by half too, while adding the same crumb to each leaves it untouched — so the difference decays toward zero. They converge to an equal share of a resource neither one can measure, without ever communicating. Fairness falls out of the arithmetic. Nobody enforced it.

Interactive · drag the ceiling, break the ½ The sender never learns the ceiling — it only ever hits it
+1.5
0.50
Current rate
0
Avg utilisation
Drops
0
Flow split
flow A flow B link ceiling (drag it) drop → halve
Leave it at ×0.50 and the average utilisation settles near 75% — the mean of a sawtooth that runs from half-full to full. That's the price of never being told the answer.
📉
Drag decrease × away from 0.50 and watch the trade. Push it to 0.90 — a timid cut — and utilisation goes up, but two flows take an age to converge and the line judders constantly against the ceiling. Drop it to 0.15 — a panic cut — and every stumble sends you to the floor: fairness arrives fast, and you waste half the link getting back. One-half is the compromise that has been humming under your internet connection your entire life.
🪜
Then look at the shape again and tell me it isn't familiar. A career: take on a bit more, a bit more, a bit more — burn out — cut back hard — creep up again. A training plan. A budget. A new friendship, feeling out how much is too much. All of them are AIMD, run by someone who was never handed the ceiling either. It also explains why you are permanently either slightly under-committed or slightly over — the algorithm has no state in which it is correct. It only has the flinch and the creep.

6The tyranny of the queue

Every AIMD story above depends on one thing: the sender finds out. A packet gets dropped, the ack doesn't come, the sender halves. Loss is the only teacher in the system. So here's a question that ruined the internet for about a decade: what happens if you make it impossible to lose anything?

Buffers were a good idea. A router gets a burst it can't forward this instant, so it holds the packets in a little queue and works through them. Bursts get smoothed, nothing gets thrown away. Memory got cheap, so engineers did the reasonable thing and made the queues bigger. Then bigger. Why would you ever want to drop a packet you could have kept?

Because a buffer that's too big doesn't prevent the problem, it hides it. An oversized queue doesn't drop your packet — it delays it. And delay is a signal TCP is not listening for. The ack still comes back, just late. So the sender concludes everything is fine and speeds up. Which fills the queue further. Which makes the delay worse. Which still isn't a loss, so the sender speeds up again. The one mechanism that could apply the brakes has been quietly disconnected, and the system happily accelerates into a wall it cannot see. This is bufferbloat, and it's why your download can be running at full speed while a video call on the same link becomes unusable — the packets aren't lost, they're thirty-first in line behind a movie.

The fix is counterintuitive and correct: drop things sooner. A queue with a hard limit outperforms an infinite one, because a full queue that drops is a queue that tells the truth. The loss is not damage — the loss is the message.

The refinement matters too. The naive rule (“drop when full,” Tail Drop) still lets a big queue sit permanently full. The better rule watches the clock instead of the count: track how long packets have been waiting, and if the minimum wait stays above a target for a sustained stretch, start dropping — even though there's room. That's CoDel, controlled delay. It doesn't ask “how many are queued?” It asks “how long has anyone been in this line?” — which is the only question that was ever about the user.

Interactive · a fast sender, a slow link, a queue Watch throughput vs. latency disagree
8
Throughput
Queue depth
0
Latency
Drops
0
queued packet dropped (the signal) buffer capacity CoDel target (100 ms)
📥
You have run this experiment on yourself. The inbox that accepts everything and answers nothing is a bloated buffer: nothing was refused, so nobody was told to slow down, so more arrived, so the queue grew — and now every message in there is late rather than declined. Throughput is unchanged (you answer the same number of emails a day). Latency is catastrophic. The cure is the same as the router's: drop early rather than promise late. A no on Monday is a delivered packet. A yes on Monday answered in April is a queue.

7Latency, not bandwidth

So here's the closing move, and it's the thing the whole chapter has been walking toward. We keep buying bandwidth — how much can flow per second. But the thing that actually ruins a conversation is latency — how long you wait for a reply. And these are not the same quantity, they are barely even related.

A famous line from the networking world: never underestimate the bandwidth of a truck full of hard drives. It's true — a van of SSDs moves more data per hour than most fibre. Its latency is nine hours. You would not hold a conversation over it. A fatter pipe does not fix a slow round trip, and no amount of capacity has ever made a queue shorter.

Which is a diagnosis of modern life, delivered by routers. We are bandwidth-rich and latency-poor. You have more channels than any human in history — mail, chat, three social networks, two work tools, a phone that is somehow also four inboxes. The throughput is astonishing. And every one of them is a queue, and everything in every one of them is late. You are not short of capacity. You are drowning in it, and the capacity is what built the queues.

The advice falls out of the chapter honestly, without anyone having to be wise about it:

  • Cap your queues. A commitment list with no limit isn't ambitious, it's bloated. Say no while the no is still cheap — for you and for them.
  • Ack quickly. A three-word reply today beats a thoughtful one in a fortnight. The gap in the acks is doing more damage than the brevity ever could.
  • Back off gracefully. Not everything that isn't working needs to be severed. Double the interval and leave the connection open.
  • Stop chasing the fourth ack. Certainty isn't available. Confident-enough is the only setting there is; attack at dawn.
  • And the punchline: the fix for being overwhelmed is a smaller buffer, not a bigger one. Every instinct you have says get more capacity, hold more, drop nothing. That instinct built the bloat. The router figured this out in 2012. We're still catching up.

8Backoff, for real

Backoff is a dozen lines and you should genuinely have it in your toolbox. The shape is a tail-recursive go over an attempt counter: run the op, return on success, return on exhaustion, otherwise sleep a random slice of a doubling-and-capped window and recurse. The randomness isn't decoration — it's the difference between a thousand clients recovering and a thousand clients re-colliding.

💛
Read backoff once more with a person in mind instead of a server. op is reaching out. Failure is them letting you down again. The function does not retaliate, does not send a final warning, and does not close the connection — it waits longer, adds a little randomness so it isn't a scheduled ritual, caps the wait so it never becomes never, and tries again. That's not resignation and it isn't naivety. It's a bounded, decaying, open-ended offer, and it's a more honest policy than most of the ones we articulate out loud.

9Check yourself

3 questions · instant feedback 0 / 3

10What's coming

This chapter had two parties who wanted the same thing and only had to work out how to say so. The next one drops that assumption. When the other side has its own agenda, the hard part isn't the channel — it's that your best move depends on their move, which depends on their read of your move, which depends on their read of your read. Recursion, but with an opponent in it. And the ugly discovery waiting there: everyone can play perfectly and everyone can still lose.

♟️
Up next · Chapter 11
Game Theory
Thinking about what everyone else is thinking about what you're thinking. Equilibria, the price of anarchy, why rational players walk into outcomes nobody wanted — and why changing the game beats playing it better.