What is a pulse test?

A marketing mix model learns from variation. A channel that has run at the same level every week for two years gives the model nothing to learn from: the posterior stays wide, and the recommendation is uncertain. A pulse test creates the variation on purpose. One channel, one change at a time, everything else stable: some weeks at the same level, some weeks off, some weeks at double or triple. The model reads the response and the range narrows.

The saturation curve says how much spend it takes to learn something, so the test is sized from the model rather than guessed. The model also prices the test: what it expects the experiment to cost against the optimal plan, perhaps a few percent fewer orders in the test weeks, in exchange for a narrower range afterwards. Dare to give the channel some gas when you first test it; a timid pulse teaches little.

Pulsing suits channels that are easy to switch: radio, Snapchat, display, out-of-home. It suits target-optimized auction channels, such as Performance Max and Meta sales campaigns, less well, because changing their budget restarts the platform's learning and muddies the reading.

A test recommendation from Odins states the channel, why, the spend pattern, the duration, the expected cost, the maximum downside and what you will learn: for example, a channel whose return is estimated at 3.2 with a range from 0.8 to 7.1, pulsed over six weeks, with the range expected to narrow to about plus or minus 1.2 afterwards. Media platforms often add ad credits when they know an MMM is measuring the test, which cuts its cost.

Pulse tests are one kind of experiment among several. Odins also takes a customer's existing lift studies and geo results into the model; what the pulse adds is a design that works at the scale of a single national market.

• Incrementality test: A controlled experiment that measures what one channel causes, by comparing exposed and unexposed groups or periods.
• Geo experiment: Run the activity in some regions and not others, then compare. The classic causal test, when the market is big enough to split.
• Confidence range: The range a Bayesian model reports around every number. A recommendation without one is hiding something.
• Saturation curve: The response curve per channel: how sales respond to spend, and where the next unit stops paying.

Go one level deeper: Recommendations and testing. Or go back to all terms in the glossary.