Hope is not a strategy

SRE started with one question, from Google's Ben Treynor: what happens when you ask a software engineer to design an operations team?

The old answer was sysadmins running services by hand — logging in, restarting things, applying configs, paging each other at 3am. It works, until you notice the hidden tax: manual ops work scales with traffic. Twice the users, twice the incidents, twice the hands on keyboards. Success means hiring forever, and your most experienced people spend their careers doing the same repairs over and over.

The SRE bet flips it. Run production with engineers who'd rather write a script than repeat a chore — and make the machines absorb the grind. The team's job isn't to do the operations; it's to automate the operations out of existence, then spend the freed time making the next thing automatic too.

Picture two teams who are both doing exactly what they're paid to do. Developers are paid to change things — ship features, run launches, move fast. Operators are paid to keep things stable — nothing should break at 3am. Both are behaving perfectly rationally.

And rationally, they go to war. Ops wants change freezes, launch reviews, and "you can't deploy on Fridays." Dev wants to route around all of it. Every release becomes a negotiation; every outage becomes a blame match. The traditional fix is more process — heavier reviews, more sign-offs — which just makes the wall taller.

SRE's fix is almost suspiciously simple: replace the argument with a single number both teams agree to in advance. Stay on the right side of it and dev ships freely; cross it and everyone pivots to reliability. We'll meet that number in section 4 — the error budget.

Here's the counterintuitive heart of the whole field. Your user reads your site on a phone with one bar of coffee-shop wifi and a battery at 4%. Their path to you drops packets, times out, and gives up far more often than your servers ever do. Past a certain point, your extra reliability is invisible — it's drowned out by the noise of the last mile.

And it isn't free. Each extra "nine" of reliability costs roughly 10× more than the last — more redundancy, slower and more cautious launches, more engineer-hours spent guarding against rarer and rarer failures. So reliability is a feature. It competes with every other feature for the same budget, and the right target isn't "perfect" — it's "reliable enough that users are happy, and not a nine more." Naming it: this is embracing risk.

Here's the trick that ends the war. Take your target — say 99.9% — and look at the other side of it. That leftover 0.1% isn't a failure you tolerate; it's a budget of allowed failure you get to spend.

Dev spends it on launches, risky experiments, and faster releases. Incidents drain it. And when it hits zero, the deal kicks in automatically: releases freeze, and everyone works on reliability until the budget refills. Nobody has to win a meeting — the budget decides. A political fight becomes arithmetic.

Three acronyms, but only one idea underneath. An SLI is just a ratio: good events over total events. "What fraction of requests succeeded in under 300ms?" That's an SLI — a number between 0 and 1 that tracks what users actually feel.

An SLO is the line you draw on that ratio: "99.9% over 28 days." It's the promise you make to yourselves. An SLA is the lawyer's version of that promise — the same kind of line, but with money attached when you miss it.

The rule of thumb that saves you: measure what users experience, not what's easy to graph. CPU load is easy to chart and tells you almost nothing about whether anyone is happy. "Did the checkout succeed?" is harder to wire up and is the only thing that matters. CPU is not an SLI; a working checkout is.

Some work is just a hamster wheel. Toil is the SRE word for it, and it has a precise smell: work that's manual, repetitive, automatable, has no enduring value, and grows as the service grows. Restarting the same stuck job. Hand-applying the same config. Clearing the same full disk every Monday.

SRE caps toil at 50% of every SRE's time — a hard ceiling. The other half must be engineering that deletes the toil. And it compounds: every chore you automate frees the time to automate the next one, so the wheel gets smaller every week instead of bigger.

The sardonic version, which is also just true: if a human is doing it twice a week, a human has already failed to write the script.

If you could only watch four things about a running service, watch these. They're the four golden signals, and most outages announce themselves in at least one of them before a single user reaches for Twitter.

• Latency — how long requests take. Crucially, measure failed requests' latency separately; a fast error and a slow success look identical if you average them.
• Traffic — how much demand the service is under (requests/sec, transactions, whatever fits).
• Errors — how often you fail, whether loudly (500s) or quietly (wrong answer, served fast).
• Saturation — how full your most constrained resource is. The thing that runs out first: memory, CPU, connections, disk.

That's the map. Here's where the rest of the course goes — one sentence each:

• Ch.2 · SLOs & Error Budgets — picking what to measure, where to draw the line, and how fast you're allowed to burn.
• Ch.3 · Eliminating Toil — finding the hamster wheels and building the scripts that delete them.
• Ch.4 · Monitoring — alerting on symptoms users feel, not on every twitchy graph.
• Ch.5 · Release Engineering — making "ship it" boring, repeatable, and reversible.
• Ch.6 · On-Call — sharing the pager without burning people out.
• Ch.7 · Blameless Postmortems — learning from outages by blaming systems, never people.
• Ch.8 · Handling Overload — graceful degradation, load shedding, and backpressure when demand wins.
• Ch.9 · Data Integrity — the only backup that counts is the one you've restored from.
• Ch.10 · Launches & Capacity — turning chaotic launches into a checklist and planning for the load you'll have.