What is a geo experiment?

A geo experiment, or geo lift test, uses geography as the control. Pick a set of regions where the activity runs and a matched set where it does not, keep everything else the same, and the difference in sales between the two sets is the activity's incremental effect. Because exposure is decided by the experimenter rather than observed, the reading is causal, and it needs no user-level tracking.

Regions that behave alike, enough of them to get a stable comparison, and enough spend to move sales above the noise. That is easy in a country with fifty states and hard in a small national market: splitting one city against another gives too few units and too much noise to read a result. Geo designs fit the US and the larger European countries; for a single small market, a pulse test that varies spend over time is usually the better design.

A geo result sits outside the marketing mix model, and its learning is encoded into it: the lift and its uncertainty become a prior on that channel's return. Odins wants to know about a planned geo test beforehand, so the design can be checked and the result can be read into the model cleanly afterwards. A test designed without the model in mind often produces a number the model cannot use.

• Incrementality test: A controlled experiment that measures what one channel causes, by comparing exposed and unexposed groups or periods.
• Pulse test: Vary one channel's spend over time, on and off or up and down, so the model can see how it responds.
• Calibration: Anchoring a model to outside evidence, usually experiment results. In a Bayesian model, calibration is a prior.
• Prior: What the model believes about a channel before it sees the data. The part of MMM that decides how much the data can teach.

Go one level deeper: Triangulation: MMM, attribution and experiments. Or go back to all terms in the glossary.