What is forecast verification?

Any model can be made to fit the past; add enough variables and the history is explained perfectly. The question that matters is whether the model predicts what has not happened yet. Forecast verification answers it the only way that cannot be gamed: train the model up to a cutoff, record its forecast for the weeks ahead, wait, and compare with what actually sold. No retrofitting.

Every Odins model is retrained monthly on data that arrives daily. Each version stores its forecast before the actuals come in, and the verification view shows the forecast line against the actual line, week by week. Because every earlier model stays available, you can also ask how the model from six months ago predicted the months since. Early models, in the first months, are typically less accurate than the model is in steady state; what matters is the direction over time, which the view makes visible.

Monitoring sits around it. The platform tracks model accuracy and raises an alert when predictions diverge from actuals, and each retrain includes a check for changes in channel efficiency, so a channel that has started working differently is caught at the next retrain rather than a year later.

Scenario planning only has value if the model behind it forecasts well. Verification is what lets a CFO plan on a scenario, and what lets a marketing team act on a recommendation even when it disagrees with their instinct: the forecast is on record, and next month will show who was right. Known external shocks, a competitor launch, a tracking bug, a weather event, are flagged in the history as noise rather than forecast, and they are the usual reason a given week misses.

• Scenario planning: Forecasting what a budget would return before committing it, and comparing several futures side by side.
• Confidence range: The range a Bayesian model reports around every number. A recommendation without one is hiding something.
• Model retrain: Data flows in daily; once a month the model is trained again on everything so far. Slow, stable movement is the point.
• Channel contribution: What each channel added to sales above the baseline over a period. The decomposition behind every ROAS figure.

Go one level deeper: How accurate is marketing mix modelling?. Staying accurate. Or go back to all terms in the glossary.