A Map of Mathematics · ch.11 · randomness has a shape
🗺️ Chapter 11 · The Frontier, part II

Randomness,
suspiciously well-behaved

Flip enough coins and patterns emerge that nobody ordered — order snaps into random graphs, primes march in pairs, and the eigenvalues of noise sync up with bus schedules in central Mexico.

1The snap

Take a pile of dots, start gluing them together with edges thrown in at random, and watch what happens to the whole. The surprise isn't that structure appears — it's how it appears. Not gradually. It snaps.

The setup is the Erdős–Rényi random graph, named for Paul Erdős and Alfréd Rényi, who studied it in 1959. Take n vertices. For each possible edge, flip a biased coin: with probability p you draw it, otherwise you don't. That's the whole model — one knob, p, the edge density. Crank p from 0 to 1 and the graph goes from a dust of isolated dots to a fully wired clique.

Now ask a yes/no question about the graph — "is it connected?", "does it contain a triangle?" — and plot the probability of yes against p. You'd expect a gentle ramp. You don't get one. You get a cliff. Below some critical density the property is almost never there; above it the property is almost always there; and the window where it's genuinely uncertain is vanishingly thin. The graph doesn't develop connectivity, it acquires it all at once — like water sitting at 0°C and then, the instant it tips over, being ice. Physicists call it a phase transition; the graph people call it a threshold. Same cliff.

For connectivity the threshold sits at p ≈ ln(n) / n — and the sharper n gets, the sharper the cliff. Scrub the slider below. Each edge has a fixed hidden value; raising p just reveals more of them. Watch the component count collapse to 1, watch "connected" flip from ✗ to ✓ — and notice how narrow the band is where it's a coin toss.

Interactive · the phase transition Drag p · regenerate the hidden values · watch it snap
Pr[connected] vs p  ·  150 sampled sequences
p = 0.00 edges: 0
threshold ln(n)/n ≈ —
Below threshold: a dust of fragments. Push p past the cliff and the whole thing wires up at once.
n = 40 vertices. The dashed line on the right marks the theoretical connectivity threshold ln(n)/n. The blue S-curve is empirical — estimated from 150 independently grown graphs at load.

Here's the 2022 punchline. For decades people proved threshold locations one property at a time, by hand, each a small saga. Then Jeff Kahn and Gil Kalai conjectured something audacious: there's a single, almost-mechanical recipe that tells you where the threshold sits for any reasonable property — it's essentially set by the cheapest "expected obstruction" you can write down. Bold, sweeping, and stubbornly unproven for years.

In 2022 it fell. Not to a famous name with a fifty-page opus — to Jinyoung Park, a postdoc, and Huy Tuan Pham, a grad student, in a proof that runs about six pages. A problem the field had circled for a decade, settled in something you could read on a train. The Kahn–Kalai theorem now pins down where every one of these snaps lives.

❄️
A threshold (sharp phase transition) means a property flips from almost-never to almost-always across a narrow band of edge density — like freezing at exactly 0°C. Kahn–Kalai, proved in 2022, says where that band sits is predictable for essentially any property.

2The lecturer nobody knew

The primes thin out as you climb — that's been known since Euclid. So the gaps between consecutive primes get bigger on average. The obvious next question is whether they ever stop getting close. Do primes keep arriving in tight pairs forever, or do the close encounters eventually run out?

"Twin primes" — pairs exactly 2 apart, like (11, 13) or (101, 103) — are the tightest possible. The conjecture that there are infinitely many of them is roughly as old as the question, and for over two thousand years nobody could prove that primes stay within any finite distance of each other infinitely often. Not 2. Not a million. Not a billion. No finite bound at all had ever been nailed down.

In 2013 it fell, and the person who knocked it over was nobody's idea of a hero. Yitang Zhang was 58. He'd finished a PhD years earlier, then vanished from academia — worked as an accountant, reportedly did a stint at a Subway sandwich shop, eventually landed an untenured lecturer post at the University of New Hampshire teaching calculus. No fame, no flashy results, no reason for anyone to watch him. Then he submitted a paper proving that there are infinitely many pairs of primes within 70,000,000 of each other — the first finite bound in history. The referees couldn't find a hole. The bound was absurd, but the wall had been breached: some finite gap recurs forever.

Then the internet happened to mathematics. Terence Tao spun up a Polymath project — a massive open collaboration, dozens of people grinding the constant down in public on a blog. Meanwhile a young postdoc, James Maynard, found a cleaner method that did the job better. Inside eighteen months 70,000,000 became 246. That's where it sits today: infinitely many consecutive primes differ by at most 246. Twin primes — gap exactly 2 — remain open. But for the first time the wall is visible, and only 244 units thick.

Sieve the primes yourself below. Slide the range and watch the histogram of gaps build. Gap 2 keeps showing up — but proving it never stops is the last, hardest 244.

Interactive · prime gap explorer Slide the range up to 100,000 · toggle the y-axis · count the twins
primes up to N = 10000
sieving…
Gap 2 (twins) keeps showing up forever? Open since antiquity. Gap ≤ 246 infinitely often? Proved, 2013–14, by a stranger and then a crowd. The orange bar is the twin-prime count.

3The physicist's shortcut

In the 1950s Eugene Wigner had a problem he couldn't solve and a problem he could fake. He wanted the energy levels of a heavy atomic nucleus — uranium, say — which means diagonalizing a giant operator (the Hamiltonian) that nobody could write down, let alone compute. So he cheated, brilliantly: if you can't know the matrix, make it random. Fill it with random numbers, respect the symmetry, and ask what the eigenvalues do on average.

The genius part: he wasn't after individual levels, he was after their spacings — how far apart neighboring energy levels sit. And the random model nailed the spacings that experiments actually measured. The headline feature is repulsion: two energy levels almost never sit right on top of each other. The probability of a near-zero gap is itself near zero. Random independent points would happily clump (that's the Poisson pattern, gaps near zero are common); eigenvalues refuse. They push apart. The curve that describes it — the Wigner surmise, p(s) = (π/2)·s·e^(−πs²/4) — starts at exactly zero and humps in the middle. Levels do not touch.

Then it got eerie. In 1972 the number theorist Hugh Montgomery was visiting Princeton and, over tea, described to physicist Freeman Dyson the pattern he'd worked out for the spacings between zeros of the Riemann zeta function — the deepest objects in the theory of primes. Dyson recognized it instantly: it was his formula, the one from random matrices. The zeta zeros repel each other exactly like nuclear energy levels. Nobody knows why the primes' hidden frequencies should obey the same statistics as a slab of uranium, but they do.

And it isn't even confined to physics and number theory. In Cuernavaca, Mexico, bus drivers had no central timetable, so each driver paid an informant at stops to report when the previous bus passed — then adjusted speed to avoid bunching. The result: the gaps between buses, measured by physicists in the late 1990s, follow the same repulsion curve. Self-spacing drivers reproduced random-matrix statistics on a city street. The pattern is so indifferent to its source that mathematicians gave the phenomenon a name — universality — and in the 2010s Terence Tao and Van Vu, and independently László Erdős, Benjamin Schlein and Horng-Tzer Yau, proved it: scramble the matrix's entries however you like and the spacing law doesn't budge. The detail of the noise is irrelevant; only the shape survives.

Compare the two worlds below. Source A drops independent points (Poisson taxis, no coordination). Source B samples honestly from the Wigner surmise (repelling levels). Watch where each histogram piles up near zero.

Interactive · the spacing fingerprint 600 samples each · regenerate · compare the left edge
A · uncorrelated (Poisson)
B · repelling (Wigner / GOE)
A hugs e^(−s); B hugs the Wigner curve
Zeta zeros: repelling. Cuernavaca buses: repelling. Poisson taxis: not. Note B's curve hits zero at s = 0 — neighboring levels refuse to touch.

4Sparse but unbreakable

Suppose you want a network where information spreads everywhere fast and there's no chokepoint — cut it anywhere and you still can't isolate a big chunk cheaply. The lazy way to guarantee that is to wire everything to everything: a complete graph has no bottlenecks because it has no gaps. But that costs a quadratic number of edges, and it doesn't scale.

An expander graph is the miracle alternative: every vertex has only a few neighbors — the edge count grows merely linearly — and yet there's still no bottleneck anywhere. A random walk on it mixes almost as fast as on the fully-connected graph. Few wires, dense behavior. The formal handle is the spectral gap: the second-largest eigenvalue of the graph sits far from the first, and the bigger that gap, the faster everything mixes and the harder the graph is to cut.

Here's the paradox the FP brain will enjoy. A random sparse graph is, with high probability, an excellent expander — randomness hands them to you for free. But writing one down deterministically, a guaranteed-good expander you can construct on purpose, turned out to need serious machinery. The first explicit optimal ones — Ramanujan graphs, built by Alexander Lubotzky, Ralph Phillips and Peter Sarnak in 1988 — came out of deep number theory, the kind tangled up with the Ramanujan conjecture about modular forms. Proving the thing has no bottlenecks meant bounding eigenvalues using tools from the heart of analytic number theory. Randomness gives them away; determinism makes you earn them.

And they earn their keep. Expanders power error-correcting codes — the same job we met in Chapter 8, protecting data from noise — because their no-bottleneck structure spreads redundancy so no local damage can erase a message. They're central to derandomization: walking an expander is a way to get "random-looking" choices using very few real random bits, which lets you shrink the randomness out of algorithms. And the subject is still alive — in 2025 a decades-old bet by Sarnak about how common the very best (Ramanujan) graphs are among all graphs was finally settled.

See the difference below: a ring and a random 3-regular expander, same number of vertices, same edge budget. Drop 300 walkers on one node and race them to uniform. The expander floods everywhere in a handful of steps; the ring is still lumpy after a hundred.

Interactive · the mixing race Step the walkers · watch total-variation distance fall
Ring · each node + 2 neighbors
TVD to uniform:
Expander · random 3-regular
TVD to uniform:
step 0
Same edge budget, wildly different plumbing. 60 nodes, 300 walkers each. Darker node = more walkers. The expander reaches near-uniform in ~10 steps; the ring stays lumpy past 100.

5The pile that organizes itself

In 1987 Per Bak, Chao Tang and Kurt Wiesenfeld asked a deceptively dumb question: what happens if you keep dropping grains of sand on a pile? Their toy: a grid. Drop a grain on a cell. If any cell ever reaches height 4, it topples — sheds one grain to each of its four neighbors and drops back to 0. Those neighbors might now hit 4 and topple too, and you get a cascade, an avalanche rolling across the grid until everything settles below 4 again.

There's no tuning. No temperature, no dial set to a magic value. You just drop grains. And yet the avalanche sizes settle into a power law: tiny avalanches constantly, medium ones sometimes, monstrous grid-spanning ones rarely — and the frequencies follow a clean straight line on a log-log plot. The system steers itself to the knife's edge where events of every scale coexist. Bak called it self-organized criticality, and the same signature turns up in earthquakes (Gutenberg–Richter law), power-grid blackouts, forest fires, and — pick your poison — the cascade when one dependency bump breaks a build that breaks twenty downstream packages.

Now the part for the functional-programming reader, because it's genuinely beautiful. The order in which you topple unstable cells does not matter. Topple them left-to-right, top-to-bottom, all-at-once, in any sequence you like — the final stable pile is exactly the same, every time. The system is confluent: like a well-behaved term-rewriting system where the normal form is independent of the reduction strategy (the Church–Rosser property, if you want the name). Local rules, no coordinator, deterministic result. The sandpile is a confluent rewrite system that happens to look like physics.

Play with one below. Click to drop grains; cells light up by height 0–3 in the five theme tints. The side panel accumulates the avalanche-size histogram on log-log axes — watch a straight line appear. Then hit the big red button and feel the power law fill in.

Interactive · the abelian sandpile Click / hold to drop grains · or unleash 5000 at once
0 1 2 3
Avalanche sizes · log-log
grains: 0 · largest avalanche: 0
Topple in any order — same final pile. The grid is confluent: local rules, no coordinator, one deterministic normal form. A straight-ish line on log-log axes is the power law showing up.

One chapter, one moral

Five fields, one idea: randomness at scale is lawful. The snaps have locations you can compute (Kahn–Kalai). The prime gaps have a finite ceiling you can prove (246). The spacings of noise have a fixed shape that doesn't care where the noise came from (universality). The expanders behave densely on a sparse budget. The piles obey an exponent nobody dialed in. None of this is "anything goes." It's the opposite — it's structure that only becomes visible once you stop looking at the individual coin flip and start looking at the distribution.

That's the punchline, and it loops straight back to Chapter 8. "Random" is not the absence of law; it's a distribution — a precise mathematical object, the one Kolmogorov nailed to an axiomatic foundation. And distributions are objects you can prove theorems about. Randomness has a shape because the shape is the math.

Where you've already met this

None of this stays in the journals. You've shipped against it.

  • SAT solvers and the satisfiability threshold. Random Boolean formulas have a sharp phase transition in the clause-to-variable ratio: below it almost all are satisfiable, above it almost none, and the hardest instances cluster right at the cliff — the exact same Erdős–Rényi snap, now deciding whether your solver hangs.
  • Load balancers as expanders. When you want requests to spread evenly with few connections and no hot spot, you want expander-like topology. "Power of two random choices" works because random sparse graphs mix fast — that's expansion doing your job.
  • Cascading failures as sandpiles. One overloaded service sheds load to its neighbors, which topple in turn — a retry storm, a dependency-bump avalanche, a grid blackout. Self-organized criticality is why the small incidents are constant and the catastrophic one is rare but inevitable.
  • The birthday paradox. That interview chestnut — 23 people, even odds of a shared birthday — is a baby threshold: a property (a collision) snapping from unlikely to likely as you cross a critical count. Same cliff, smaller graph.

6Check yourself

3 questions · instant feedback 0 / 3

What this era unlocked

This chapter taught one habit: when something looks random, don't shrug — measure its distribution and look for the law. Snaps have locations, spacings have shapes, piles have exponents. The next chapter takes the same move and points it at your data. It turns out a cloud of points has a shape too — holes, loops, dimensions — and there's a barcode scanner for reading it off.

🫧
Up next · Chapter 12
The Shape of Data
Grow balloons around your data and read off its holes, reconstruct a signal from a fraction of the samples, and move probability around like dirt — geometry gets a job in data science.