How often to refit a marketing mix model, and why

Key Takeaways

• • A media mix model describes the history it was given. New weeks, moved budgets and changing markets all make an old fit out of date.
  • Vendors and open-source tools differ on cadence. Some refresh weekly, and some support monthly, weekly or even daily updates.
  • Refitting also gives you a running test: you can compare what the last fit predicted with what happened.
  • Every refit should re-run its data checks and convergence checks, and it should be allowed to fail.

A media mix model is a statistical summary of the weeks you supplied, and the business keeps generating new weeks. This page addresses the questions that follow. How often should you refit a marketing mix model, what changes when you do, and which checks should accompany each refit?

Why isn't a media mix model run once?

A fitted model is a snapshot of the data that existed when it was estimated. Each week adds an observation, and your own budget changes move channels to new positions on their response curves. A model that is never refit gradually describes a business that no longer exists, so you would be deciding today's budget with last year's picture.

Three forces make an old fit stale. New observations change the estimates and narrow the intervals, and market conditions shift, so a channel that worked last year may respond differently now. Your own decisions matter as well. Once you reallocate budget, the channel occupies a different position on its saturation curve, and its marginal return changes accordingly. I explain that curve in adstock and saturation explained.

None of this means the earlier fit was wrong, only that it was answering a question about a period that has ended.

Four things change from one refit to the next

Four quantities change from one refit to the next: the intervals, the point estimates, the seasonal shape and the ranking of channels. Intervals usually narrow as observations accumulate, and estimates move when the market or your allocation changes. A second year of data lets the model observe a seasonal cycle repeating, and rankings shift whenever the underlying estimates do.

The next example is synthetic and illustrative, not real customer data. One channel's marginal return is estimated at each of eight refits. For the first four, the estimate hovers near $1.10 with wide intervals. Then a market change hits, and the next four refits track the estimate down to about $0.66. Analysts call such movement between successive fits drift.

Figure 1. One channel's marginal return re-estimated across eight refits.

A model estimated once would continue to report the first value, whereas the refit sequence reveals the decline as it happens. By the eighth refit, the interval sits almost entirely clear of the one from the fourth, so the change exceeds sampling noise. That is the case for a schedule, and it is also why I read marginal return as a number with a date attached.

How often to refit a marketing mix model

There is no universally correct cadence, because the appropriate rhythm depends on how often you make budget decisions and how quickly your markets change. Published recommendations range from weekly refreshes to monthly or even daily ones, so it helps to examine what each source says and to choose deliberately.

In 2023, Michael Kaminsky argued for weekly refreshes in "Why should we refresh our MMM every week?" He wrote it for Recast, a vendor whose product refits weekly, so weigh his case with that interest in mind. His reasons are still worth listing. Weekly updates avoid end-of-quarter surprises and make results actionable while a campaign is in flight. They also create feedback loops that let you verify the model.

Meta's Robyn documentation takes a more flexible position, and its guide says the refresh function lets an MMM act as a continuous reporting tool. Its feature page says the refresh enables reporting as frequently as the data allows. The frequency can be monthly, weekly or even daily, and the page contrasts this with conventional bi-annual cycles.

My own rule is to refit at least as often as you reallocate budget. Do not wait so long that a market change slips past for a quarter. That is a rule of thumb and not a published standard.

Does a refit start from scratch?

It depends on the tool, so ask. Some systems re-estimate the whole model on all history each time. Others narrow the search around earlier results, which keeps estimates stable while new weeks are fit well. Both are legitimate designs, and they trade stability against openness to change.

• What is re-estimated? — Recast (its documentation): The entire model, for all of history, each week; Robyn refresh (its documentation): A model for the new period, with hyperparameter bounds stabilized from the initial model
• What is the fit judged on? — Recast (its documentation): Not stated on the page; Robyn refresh (its documentation): Error on the new period only, because the aim is to describe it better
• What does it hold steady? — Recast (its documentation): Not stated on the page; Robyn refresh (its documentation): Expects a stable baseline compared with the initial model
• Stated caveat — Recast (its documentation): None found on the page I read; Robyn refresh (its documentation): "Not yet tested thoroughly and might provide instable refresh results"

Sources: Recast knowledge base; Robyn "Key Features" and analyst's guide. The table compares what each page says and does not rank the tools.

Recast's documentation says that each week it does not run only the last week of data. It re-estimates the entire model for all of history. Robyn's refresh function works differently, and its guide says it stabilizes hyperparameter bounds based on the initial model selection.

Robyn's own feature page adds a caveat that matters for anyone planning to lean on it. The page says the feature is not yet tested thoroughly and might provide instable refresh results. I take that as a reason to compare a refreshed model against a full re-estimate before relying on it.

Neither design is wrong, but the question to put to any provider is what changes when a refit runs and what is held fixed. If the baseline is held fixed, for example, a genuine baseline shift will take longer to appear. I describe the baseline in MMM baseline, seasonality and promotions.

Every refit should repeat the same checks

Every refit should repeat the checks that a first fit requires. The data should still meet the model's requirements, sometimes called a data contract, the sampler should converge, and the decomposition should remain plausible. A refit that skips these checks can quietly publish an estimate that would have been rejected on day one.

For convergence, the Stan reference manual notes that Vehtari and colleagues suggest a general R-hat threshold of 1.01. The check is covered in more depth in Bayesian vs frequentist MMM. A scheduled refit should apply that modern check each time.

Five items belong on the list for each update:

1. Did the data arrive complete, with no missing weeks and no negative spend from refunds or credits?
2. Did the sampler converge, and are diagnostics recorded for the run?
3. Are the new intervals consistent with the old ones, or has something moved beyond noise?
4. Does the baseline share still look plausible?
5. How did last period's forecast compare with what actually happened?

The last item is the running test that a schedule makes possible. Kaminsky's post makes the same point: continuous updates let you verify whether the model's predictions held up. In 2024, Google researchers Yingxiang Zhang and colleagues also showed how experiment results can enter a model as priors. That gives an outside check on each refit.

A backtest: refitting every four weeks cut forecast error from 17.7% to 14.8%

A backtest, also called a holdout forecast check, withholds the most recent weeks from the estimation and measures how accurately the model predicts them.

The series has one channel whose marginal return is 1.10 until week 80 and 0.66 afterward, a market change. Two models predict weeks 91 to 114. One is fit once on weeks 1 to 78 and never updated. The other is refit every four weeks on all history to date. Average weekly revenue in the window is about $260 thousand.

• Fit once on weeks 1 to 78 — Slope estimate: 1.02; Mean absolute percentage error, weeks 91 to 114: 17.7%
• Refit every 4 weeks on all history — Slope estimate: 0.83 to 0.96; Mean absolute percentage error, weeks 91 to 114: 14.8%

The fit-once model over-forecasts by about $46 thousand a week, because it still assumes the earlier return. Refitting closes part of the gap, from 17.7% to 14.8%. It does not close all of it, because the estimates stay above the true 0.66 while the older weeks still weigh on each fit.

Two lessons follow. A schedule pays for itself in forecast error, and a refit that leans on all history is slow to follow a structural break. That is why the check in item 3 above matters, and why comparing a refit with an outside experiment is worth the effort.

Should a refit be allowed to fail?

Yes. A refit that cannot produce a defensible fit should say so, and it should leave the earlier fit clearly dated. Publishing a fresh but unreliable set of numbers is worse than showing a dated one with a notice. A model that always answers on schedule is not the same as a model you can trust.

Failure usually has ordinary causes. A refund posted as negative spend can break the arithmetic, and a gap in the weekly sequence can break the time series. A channel that stopped varying can leave the model with nothing to learn from. In each case the informative output is a stated reason and a remedy.

The pillar guide makes the broader case that a model able to refuse is easier to defend to a boss or a client.

Frequently asked questions

Is weekly refitting always better than monthly?

No. Weekly refits suit teams that reallocate weekly and maintain stable, complete data feeds, while monthly refits can suit slower decision cycles and smaller data volumes. Choose the cadence that matches your decisions, then verify that each update runs its checks. Frequent updates only help when the underlying data is trustworthy.

Will my results jump around if I refit often?

Estimates can move, but movement inside the credible interval is ordinary sampling variation. Look for changes that leave the interval and persist across several consecutive refits. Tools that stabilize hyperparameters between refreshes reduce volatility, at the cost of responding more slowly to genuine change.

What if a refit disagrees with the last one?

Compare the intervals first. If they overlap heavily, the difference is probably noise. If they do not, look for a cause such as a market change, a budget move, a data correction or a new promotion. Then check the disagreement against an experiment, if you have one.

Do I need new data before a refit?

Yes, at least a new week, because refitting the same rows would return the same answer within sampling noise. The value of a schedule comes from new observations, corrections to old ones, and updated controls such as a promotion calendar or a new channel.

Sources

Every external claim above comes from the pages listed here, each opened on the retrieval date shown.

• Recast, Kaminsky, "Why should we refresh our MMM every week?"
• Recast, "Integrate Recast into your weekly marketing workflow," Recast MMM Knowledge Base,
• Meta, "Key Features," Robyn documentation
• Meta, "Analyst's guide to MMM," Robyn documentation
• Stan Development Team, "Stan Reference Manual: Posterior Analysis"
• Google Research, Zhang et al., "Media Mix Model Calibration With Bayesian Priors"