Wanamaker's Lament

Law 07 / 20 Measure What Matters

Some advertising money buys sales that would have happened anyway, and the hard part is finding out which spend actually causes results.

  • Adage or reasoning rule
  • 5 min read
  • Documented study
  • Reviewed by Ali Dezhami, November 8, 2026

The dilemma

Illustrative scenario

Dana is e-commerce director at an outdoor-gear retailer. Ad costs have risen for three years, and the CFO arrives at the budget meeting quoting the famous line about half of advertising being wasted. His proposal: cut every paid channel by 50% and see what happens.

The agency pushes back with its dashboard. Every channel shows a positive return on ad spend, and two look outstanding: ads on searches for the retailer's own name, and retargeting ads shown to people who left items in their cart.

Dana is uneasy about both camps. The retailer already appears first in the free search results for its own name, and the retargeted shoppers had chosen products before they ever saw an ad. The conflicting metric: the platforms report excellent returns, but nobody has measured how many of those sales would have happened without the ads. Cutting blindly or keeping everything both look like guesses.

The short answer

Wanamaker's lament is the complaint usually quoted as "Half the money I spend on advertising is wasted; the trouble is I don't know which half." It is commonly attributed to the Philadelphia retailer John Wanamaker, but the attribution is unproven: Quote Investigator treats Wanamaker as the leading candidate, with the earliest full match found in print in 1919 and partial matches back to 1890.

How strong is the evidence?

Adage or reasoning rule

A memorable saying or reasoning rule, not an experimental finding. Treat it as a prompt to check, not as evidence.

This is an adage, not a finding. Nobody measured that half of advertising is wasted; the line survives because it names a real problem: it is hard to know which spending causes sales.

Research explains why. Lewis and Rao (2015) analyzed 25 large field experiments run with major retailers and financial brokerages and showed that individual purchases vary so much, relative to the typical effect of an ad, that even very large experiments produce wide uncertainty. They report that the confidence intervals around return on investment were very wide, with a median interval of more than 100 percentage points. Ordinary dashboards, which do not compare against a control group at all, are in a weaker position still.

Platform attribution adds a second problem: it credits conversions to ads that people saw or clicked, whether or not the ad changed their behavior. Shoppers who were already going to buy, such as people searching your brand name or returning to a full cart, are exactly the ones these reports reward most.

The useful reading: the share of wasted spend is unknown and varies by channel, audience and time. It could be far less than half or more. The way to reduce the uncertainty is to measure incrementality with holdout or geographic experiments.

Investigating an example

Documented study

What happened when eBay paused brand search ads

Blake, Nosko and Tadelis ran large-scale field experiments on eBay's paid search advertising. When eBay stopped buying ads on brand keywords (searches containing "eBay"), users largely reached the site through the free organic links that appeared for the same searches, and the authors report that brand keyword ads had no measurable short-term benefits. For non-brand keywords, the measurable effect was concentrated among new and infrequent users, while frequent users, who would likely have come anyway, were little affected.

Limits: eBay was an exceptionally well-known brand that ranked at the top of organic results for its own name, and the study measured short-term effects in one market and period. Brand search can behave differently when competitors bid on your name or when your organic listing is weak. The study shows how attribution can overstate effects; it does not show that search advertising never works.

A published study or record, linked below. Its limits are stated.

Choose your move

You are in the dilemma above. Which move would you make? Open each one to see its trade-offs.

A Cut every channel by 50%, as the CFO proposes.

Saves money at once, but it cuts effective and wasteful spend alike. Some damage may appear only months later, and you still will not know which half was wasted.

B Keep everything, because every channel shows a positive platform return.

No disruption, but platform returns include sales that would have happened anyway, especially in brand search and retargeting. You may keep paying for customers you already had.

C Run holdout tests on the largest, most doubtful lines first, then reallocate on incremental results.

The best route to learning which spend works, but it is slower, forgoes some sales in the holdout groups, and tests can come back inconclusive. It needs enough volume, a set duration and patience.

No move wins in every business. The law tells you what to check, not what to choose.

How to apply it

  1. Rank spend by size and doubt. List paid lines by budget and by how likely their conversions are to happen without the ad, such as brand search or retargeting of cart abandoners.
  2. Design a holdout. Randomly withhold ads from a group of users or regions, and fix the sample size and duration before you start.
  3. Measure the difference. Compare conversions, revenue and margin between exposed and held-out groups, with a confidence interval.
  4. Calibrate attribution. Divide incremental conversions by platform-attributed conversions for the same period; use the ratio to discount future reports.
  5. Reallocate and re-test. Move budget toward lines with proven incremental return and repeat tests periodically, because effects change.

What to measure

Incremental conversions and margin from each test, the ratio of incremental to platform-attributed conversions, and the confidence interval around each estimate. Guardrail: decide in advance what result would change the budget, and do not read an inconclusive test as proof that an ad has zero effect.

Worked example

Worked example: a retargeting holdout test (illustrative numbers)
MeasureResult
Conversions attributed by the ad platform 300
Exposed group: 50,000 users, conversion rate 1.20%
Held-out group: 50,000 users, conversion rate 1.08%
Incremental conversions (difference x 50,000) About 60
Share of attributed conversions that were incremental About 20%

Illustrative numbers, not data from a real test. Equal group sizes are used to keep the arithmetic simple. A real test needs a sample size worked out in advance, and with differences this small the confidence interval can be wide.

Where it breaks

  • It is an adage. There is no measured 50% waste rate; the real share varies and is often unknown.
  • The attribution is disputed. Wanamaker is the most likely source, but the quote has not been confirmed in his own words.
  • Unmeasured is not the same as wasted. Long-term brand effects are poorly captured by short tests.
  • Experiments have limits too. Noise, short horizons, small samples and spillover between groups can blur results.
  • One study does not transfer automatically. The eBay findings depend on eBay's brand strength and search position.
  • Ethics: holdouts are fine for advertising, but do not withhold service messages, safety notices or information customers are owed.

What if you apply it incorrectly?

A CFO takes the quote literally and orders a 50% cut, split evenly across channels. Brand search and retargeting costs fall with little lost revenue, but so does the prospecting that was bringing in new customers. Six months later new-customer numbers are down, and nobody can say which cut caused it, because no holdout was kept.

Myth versus evidence

The social-video version

Exactly half of all advertising is wasted, as Wanamaker proved.

What the evidence supports

The line is a memorable quote of uncertain origin about uncertainty; the wasted share is unknown and has to be measured with experiments.

Five-minute field assignment

Pick your largest paid channel and note last month's platform-reported conversions. Estimate what share of those buyers had already visited, searched your brand name or added items to a cart before seeing an ad. Then sketch a holdout test: who is held out, for how long, and what result would make you change the budget.

Checkpoint

Two questions: one about the idea, one about a decision. Answer both correctly to master this law. Questions are drawn from a larger bank and graded on our server; a retry brings different questions.

Evidence ledger

Every factual claim in this lesson, where it comes from, and how far it has been checked.

Further reading

Which other law could explain this decision?

Editorial review: Ali Dezhami, November 8, 2026. Last substantive update: November 8, 2026. Our methodology