Find your most incremental channel with geo holdout testing

The quick context

A North America wide pet adoption platform ramped media spend year over year, but conversion volume barely moved. In one month, spend rose almost 300 percent while conversions increased only 37 percent.

Sound familiar? Here is the thing. Platform reported efficiency does not equal net new growth. You need to measure incrementality.

The core insight

Run a geo holdout test to measure lift by channel. Then compare cost per incremental conversion and shift budget to the winner.

In this case, the channel that looked cheaper in platform reports was not the most incremental. Another channel delivered lower cost per incremental conversion, which changed the budget mix.

The measurement plan

The three cell geo holdout design

Cell A, control, no paid media. This sets your baseline.
Cell B, channel 1 active. Measure lift versus control.
Cell C, channel 2 active. Measure lift versus control.

Why this matters. You isolate each channel’s true contribution without the noise of overlapping spend.

Pick comparable geos

Match on baseline conversions, population, and seasonality patterns.
Avoid adjacency that could cause spillover, like shared media markets.
Keep creative, budgets, and pacing stable during the test window.

Power and timing

Run long enough to reach statistical confidence. Think weeks, not days.
Size cells so expected lift is detectable. Use historical variance to guide sample needs.
Lock in a clean pre period and test period. No big promos mid test.

What to measure

Primary, incremental conversions by cell, lift percentage, and absolute lift.
Efficiency, cost per incremental conversion by channel.
Secondary, quality metrics tied to downstream value if you have them.

What we learned in this case

Top line, channel level platform metrics pointed budget one way. Incrementality data pointed another.

Paid social outperformed paid search on cost per incremental conversion. That finding justified moving budget toward the more incremental channel.

Turn insight into action

A simple reallocation playbook

Stack rank channels by cost per incremental conversion, lowest to highest.
Shift a measured portion of budget, for example 10 to 20 percent, toward the best incremental performer.
Hold out a control region or time block to confirm the new mix keeps lifting.

Guardrails so you stay honest

Use business level conversions, not only platform attributions.
Watch for saturation. If marginal lift per dollar falls, you found the curve.
Retest after major changes in market conditions or creative.

How to read the results

Calculate the right metric

Cost per incremental conversion equals spend in test cell divided by lift units. This is the apples to apples way to compare channels.

Check lift quality

Are the incremental conversions similar in value and retention to your baseline? If not, weight your decision by value, not by volume alone.

Look at marginal, not average

Plot spend versus incremental conversions for each channel. The slope tells you where the next dollar performs best.

Common pitfalls and fixes

Seasonality overlap, use matched pre periods and hold test long enough to smooth spikes.
Geo bleed, pick non adjacent markets and monitor brand search in control areas for spill.
Creative or offer changes mid test, freeze variables or segment results by phase.

The budgeting loop you can run every quarter

Measure, run a geo holdout with clean control and separate channel cells.
Find the lever, identify which channel gives the lowest cost per incremental conversion.
Test the shift, reallocate a slice of budget and watch lift.
Read and iterate, update your mix and plan the next test.

What this means for you

If your spend is growing faster than your conversions, you might be paying for the same customers twice.

Prove which channel actually drives net new conversions. Then put your money there. Simple, and powerful.