Should we build measurement in house or buy it?
Our answer
Buy the measurement tools and keep the judgment in house. Rent dashboards and models until a named person can maintain a build. Own the definitions, the weekly read, your raw data and the tests that check vendors.
- Confidence
- Medium
- Strongest proof
- 4 practitioners
- 9 opinions or claims
- Sources
- 9 sources
- 13 statements
PA
BMM
New here? Take the 1-minute tour
Why this is the answer
3 reasons and our judgmentPeople, not licenses, drive the cost. Eightx's 24-month model for a $6M brand ran from $11,976 for a managed dashboard to $32,385 for a freelance-built stack, and it says labor drives the gap.
A homegrown mix model takes a team and up to a year. BlueAlpha puts a serious in-house mix model at 2 to 4 full-time people and 6 to 12 months. Haus describes in-house builds hitting a build or buy decision around month twelve.
The judgment still belongs in house. Eightx keeps analytics work close to the profit and loss statement in house as brands grow, and MarSci advises checking any model's biggest claim with an incrementality test.
Our judgment
- Buy rather than build until a named person can own the data. Eightx's model shows labor rather than license deciding the cost and puts the trigger at having that person, and BlueAlpha and Haus describe in-house mix models stalling at upkeep.
- Keep the metric definitions, the weekly read and the tests that check any vendor in house, even when everything else is bought. Eightx says analytics work sits too close to the profit and loss statement to outsource at scale, and MarSci and Michael Kaminsky make checking a model against a test the first job.
- Polar Analytics's view that buying wins at every size doesn't change the answer. Polar Analytics sells the platform it recommends, and Eightx and BlueAlpha describe builds that can pay once the people exist.
- Keep your raw data exportable under any contract. Eightx says modeled views and attribution history do not export cleanly when you cancel a managed tool.
Our judgment, built from Eightx, BlueAlpha, Haus, MarSci, Michael Kaminsky and Polar Analytics: our reading of these sources, not a finding from any one of them.
What the sources say
13 statements from 9 sourcesWorkMagic argues in-house geo lift tests are cheap to code but hard to do well: validate the design with a placebo test before spending, know each test's minimum detectable lift, and stress-test the readout.
“Every geo test has a minimum detectable lift: the smallest effect it can reliably tell apart from noise. That threshold implies a budget floor.”
A Haus scientist says a typical in-house mix model takes weeks to stand up, but clashes with tests lead teams to tune priors, and around month twelve they must fund a team or pay an outside provider.
“And prior tuning is essentially cherry-picking.”
MarSci, which sells a managed mix model on Meridian, says build only if modeling is close to your core business and you can staff it; most mid-sized teams should buy. Either way, check it with an incrementality test.
“Meridian and Robyn are free. The other 90 percent of the project is not.”
mbuzz, an attribution vendor, says free Robyn and Meridian still need a modeler who costs more a year than most paid mix tools, plus 12 to 24 months of consistent weekly data. Under 12 months, run an experiment.
“Both are free. Neither is cheap.”
Recast's co-founder says free open-source mix models beat an analyst's ad hoc regression, but some paid tools just run Meridian behind attribution dashboards. First decide how you will check the model is correct.
“is how are we going to evaluate if the model is correct or not”
Eightx models 24-month reporting costs for a $6M brand from about $12,000 to $32,000 by tool, and says labor drives cost, so buy until someone can own the data.
“The trigger is having the person, not hitting a revenue number.”
Eightx says an $8M DTC brand typically spends $80,000 to $120,000 a year on 15 to 22 software tools, and to replace a tool with a hire only when it costs over 80% of a hire and fills 30-plus hours a week.
“Most flagged tools get right-sized, not cancelled.”
Polar Analytics, which sells an analytics platform, says a custom data stack takes 6 to 9 months and $15k to $38k a month before salaries, worth it only with a team and unusual data needs.
“It is the right call only when you have the team and a data model no vendor can express.”
Matt Bertulli built AI tools and dashboards that pull live ad account data, but says he lost days debugging and shipped tools that broke the next day.
“the prototype is not the product.”
Eightx, a CFO firm citing vendor materials, says SaaS mix modeling for mid-market DTC runs roughly $30K to $80K a year, and free open-source tools cost about $40K to $120K of senior analyst time to build.
“Recast specifically signals roughly $1.5K-$4K/month (about $18K-$48K/year).”
Eightx, a CFO firm, says a marketing analyst usually appears at $15M to $20M in annual revenue; before that the head of growth runs the spreadsheets. Analytics should move in-house by $15M to $25M.
“Marketing analytics or ops usually appears at $15 to $20M, before that it's the head of growth running spreadsheets at 11pm.”
BlueAlpha, which sells mix models, says an in-house model takes 2 to 4 full-time people and 6 to 12 months, worth it mainly above $100M in media with a data science team.
“The model tends to live in one person's head, and when that person leaves, its credibility leaves with them.”
Polar Analytics, which sells analytics software, puts an analyst at $90,000 to $150,000 a year fully loaded and says DTC teams can now run analytics without one, except for mix models and formal incrementality tests.
“The boundaries are MMM, formal incrementality testing, and truly novel statistical work.”
In the full answer
For readers who sign in- What to do this week3 steps
- The numbers behind it3 figures from cited sources
- Where each source standsA chart placing each source on the question
- The strongest disagreementThe best case against the answer, from a cited source
- How sure we areWhat its confidence rests on, and what would change the answer