Attribution hands out credit for the sales it can see. Incrementality asks the ruder question — how many of those sales you'd have gotten for free. This chapter is the war between those two answers, and the armistice that ends it.
Here's the whole chapter in one line: credit is not causation — the only way to know what your spend caused is to compare against a world where you didn't spend, and everything short of that comparison is storytelling with decimals. The rest is how to buy that comparison, and what to do with the dashboards in the meantime.
John Wanamaker, department-store magnate, is credited with marketing's most durable joke: "Half the money I spend on advertising is wasted; the trouble is, I don't know which half." For a century that line was accepted fate — you bought the page, you rang the register, and the connection between them was a matter of faith and folklore.
Then digital arrived and promised to end the joke. Every impression logged. Every click trailed. Every conversion beaconed back to the ad that "drove" it. Two decades and trillions of logged events later, here is the strange part: the honest answer to "which half" is still contested — inside the most instrumented industry in history.
The reason is that Wanamaker's question was never a data question. It's a counterfactual: what would have happened if you hadn't spent? And no log file, however complete, contains the world where you didn't run the ad. A programmer knows this shape: you can't diff against a branch you never ran. If you want the comparison, you have to construct it — deliberately, expensively, by withholding the treatment from someone.
The click trail tells you who bought after seeing your ads. It cannot tell you who bought because of them. "After" is free and everywhere. "Because" costs a control group. Everything in this chapter follows from the gap between those two words — a gap Chapter 15's coupon-lineage measured with holdouts a century ago and click-land spent twenty years forgetting.
An attribution model is a rule for dividing conversion credit among the touches you observed. A customer saw a display ad, watched a social video, clicked a brand search result, clicked a cart email, and bought. Who gets the sale?
Notice what changing the model does: it redraws the map without adding territory. Same touches, same one purchase — the credit moves, the knowledge doesn't. And every member of the family inherits the same congenital flaw: exposure is not random. Ads are aimed at people judged likely to buy; retargeting literally selects people who carted your product yesterday. When the targeting works, the ad's audience was already converging on the register — and every model in the tree hands the ad credit for gravity.
Programmer's version: attribution is feature importance computed on correlated features. The retargeting pixel isn't a cause of purchase; it's a feature that leaks the label. Any model that only sees the logged journey will reward the leak — elegantly, reproducibly, and wrong.
Try it. One journey, five models — then run the test none of the models can imagine.
The second structural problem isn't the models — it's who runs them. Each ad platform measures its own conversions, with its own pixel, under its own rules, and reports its own grade. Three habits of that self-grading are worth knowing by name:
Click windows. A platform typically claims any purchase within some window of a click — seven days, twenty-eight days. Buy a fortnight after clicking anything, and that click's owner books the revenue.
View-through credit. Stricter still: many defaults claim purchases that happen within a day of an ad merely being seen. Scroll past a promoted post at breakfast, buy at lunch because your friend recommended it last month — the platform's dashboard records an ad-driven conversion, no click required.
Overlapping juries. Every platform runs these rules independently, over the same customers. One purchase can sit inside a search click window, a social view window, and an email click window at once — and each dashboard counts it, whole.
The result is an arithmetic scandal hiding in plain sight: sum your platforms' reported conversions and you will routinely have "sold" more than you sold. Each number is defensible under its own rules; the total is fiction. And the incentive gradient underneath is not subtle — the party reporting the grade is the party selling the next semester's tuition.
So how do you buy the word "because"? You reintroduce the thing the click trail skipped: a group that didn't get the ad. Incrementality is just that comparison, and it comes in three practical sizes:
User-level lift tests. The platform randomly withholds your ad from a slice of the target audience. The cleanest versions use ghost ads: the system records the auctions your ad would have won in the control group, without serving it — so both groups have identical targeted intent, and the difference in purchases is your lift, not your targeting. This is the experiment attribution silently pretends to be.
Geo experiments. Randomize markets instead of people: pause or boost spend in matched regions and read the difference in actual sales. Coarser, slower — and unkillable, because they need no user tracking at all. As the tracking substrate erodes (Chapter 33's story), the geo test has quietly become the gold standard again: Hopkins' split run, upgraded with better statistics and worse weather.
Always-on holdouts. A small permanent control — a few percent of users or one quiet market — that never sees a given channel. Less precise than a designed test, but it turns incrementality from an annual event into a dashboard of its own.
And what do these instruments keep finding, decade after decade, platform after platform? The same verdict: attributed ROAS overstates incremental ROAS, often by multiples — and the overstatement is worst exactly where the targeting is sharpest. Retargeting and brand-term search sit at the top of the flattery league every time, for the reason Section 2 predicted: the sharper the targeting, the more the audience was coming anyway. (Chapter 15 told you what eBay found when it finally ran the holdout on brand search; that result was a preview of this entire chapter.)
At this point the tempting conclusion is: attribution lies, delete the pixel. Wrong lesson. Attribution has three virtues no experiment can match — it's cheap, it's instant, and it's always on. Experiments are the opposite: slow, expensive, and chunky. You get a lift number per quarter, not per creative per hour.
So use each instrument for what it is. Attribution is a speedometer, not an audit. It is genuinely good at questions where its biases hold still: which of two creatives is pulling harder in the same channel, at the same funnel depth, on the same audience; whether a campaign died overnight; whether the tracking itself broke; how spend is pacing against plan. In those comparisons the intent-correlation cancels out, and the cheap, instant number is the right tool.
Where it fails is exactly where the stakes are highest: comparing across channels with different intent profiles. Retargeting versus prospecting video is not a fair race in click-land and never will be — one of the runners started at the finish line. Chapter 19 showed what happens when that race sets the budget: the dashboard illusion, institutionalized.
The operating rule fits in one line: attribution compares like with like; experiments compare against nothing at all — and only the second answers "was it worth it?"
The measurement war ends not with a winner but with a treaty, and the treaty has a name the industry has mostly settled on: triangulation. Three instruments, three questions, three altitudes:
The triangle's edges matter as much as its corners: experiments calibrate the MMM (a lift test pins the model's curve for a channel to reality), the MMM allocates across channels, and attribution steers inside them. When the three disagree — they will, constantly — the disagreement is information about altitude, not error: the speedometer, the map, and the odometer are describing the same trip at different scales.
Run a campaign through all three lenses and watch them argue.
Compress the war into a working protocol and it's four moves, run on a calendar:
That last move is the real war. Attribution numbers are big, flattering, and arrive daily; incrementality numbers are small, rude, and arrive quarterly. Every incentive in the building points at the big number — the agency's fee justification, the channel manager's bonus, the platform's next pitch. The organizations that get this right aren't the ones with the fanciest models; they're the ones where the CFO heard the smaller number from marketing first.
One instrument in the triangle still needs its story told: the old econometric workhorse that measures marketing without tracking anyone — dusted off, re-fitted, and suddenly the most fashionable model in the building. Next chapter.