Learn

Should I use MMM or incrementality testing?

Short answer

Both. MMM answers 'what is each channel worth and how should I reallocate the whole budget'. Incrementality testing answers 'is this specific activity actually causing lift'. The best measurement systems calibrate MMM with a rolling programme of incrementality tests.

What each method is for

MMM is strategic and continuous: total-portfolio ROI, saturation curves, budget optimisation, scenario planning. Incrementality testing is tactical and episodic: a controlled experiment (geo-holdout, PSA test, switchback, conversion lift) that measures the causal effect of one specific activity in one specific window.

Why one is not enough

MMM has identification problems - correlated spend, macro shocks and platform changes can bias any single channel's coefficient. Incrementality tests solve identification with randomisation, but only for one channel at a time, and only for the time and geography of the test. Use tests to anchor MMM priors and MMM to generalise test findings to the full portfolio.

The calibration loop

Run 4-8 incrementality tests a year across the biggest and most contested channels. Convert each measured lift into a Bayesian prior on the corresponding MMM coefficient. Refit the model. Track the drift between the model and the experiments over time - that drift is your measurement health signal.

When to pick just one

If you only run performance channels and have no offline media, a strong incrementality-testing programme plus platform diagnostics may be enough. If you have TV, sponsorship or complex offline, MMM is required. Almost every serious advertiser ends up with both.

See how twenty10 puts this into practice

Bayesian MMM, calibrated with experiments, refreshed monthly, delivered as a decision system.

Explore our solutions