All answers

The Prevailing
Answers for
Business model:
Business model, pick up to two

How do we design a lift or geo test for our order volume: how big a holdout, how long, and what pass bar do we set before launch?

The PrevailingUpdated Nov 17, 2026

Our answer

Set the pass bar at your breakeven lift before launch, then size the test to detect it. For a geo test, hold out at least 15% of markets, run four weeks or more and read once at the end.

Confidence
Medium
Strongest proof
3 controlled tests
5 Meta rules, 6 practitioners, 7 opinions or claims
Sources
16 sources
21 statements

New here? Take the 1-minute tour

Why this is the answer

3 reasons and our judgment

Set the pass bar at your breakeven. Measured, an incrementality testing firm, advises a target lift at breakeven and a test whose minimum detectable effect is at or below it. Phil Kiel's agency agreed a 1.5 incremental ROAS bar near breakeven before its Haus holdout.

Small holdouts can't read a result. Elevarus, a paid media agency, calls a holdout under 15% of geos statistically useless, and Sellforte, a marketing mix modeling firm, says holding out 30% to 50% of eligible geographies often works better.

Short tests miss lagged sales. Measured says almost no test should run under four weeks. In its example, a 10% lift at about 5,000 orders a week takes about six weeks to detect, and at 1,250 orders a week more than 20 weeks.

Our judgment

We drew one rule from several: hold out at least 15% of markets and often 30 to 50%, with at least 12 in each group, using Darkroom's 12 to 15 markets a side and Phil Kiel's test, which held out 50% of the UK. Amsive's 20 or more per arm for national channels is the high end. We count a post-test window for purchase lag inside the four weeks, so Ridge's and Kiel's three weeks plus a post-test week meet it, which keeps this answer in line with the large-account answer to How often to run incrementality tests. Judge the confidence interval against the bar once, at the planned end. Lewis and Rao's finding that fine differences in return need many times more sample is why the bar should mark the decision boundary, not a precise target.

Our judgment, built from Elevarus, Sellforte, Phil Kiel, Darkroom, Amsive, Measured, Eightx, Recast, Ridge, Meta and Lewis and Rao (Quarterly Journal of Economics, 2015): our reading of these sources, not a finding from any one of them.

What the sources say

21 statements from 16 sources
  • MetaUndatedSupports

    Meta advises against using the post-test window to calculate Conversion Lift results, saying it can add variance and shrink the measured effect once holdouts end.

    “While we offer the option, we do not recommend using the post-test conversion window for calculating results.”

    Read it on facebook.com

  • MetaUndatedSupports

    Meta says lift tests automatically create test groups who could see your ads and holdout groups who are intentionally withheld from them. The page gives no holdout share.

    “Lift tests will automatically create test and holdout groups”

    Read it on facebook.com

  • WorkMagicNov 20, 2026Supports

    WorkMagic argues in-house geo lift tests are cheap to code but hard to do well: validate the design with a placebo test before spending, know each test's minimum detectable lift, and stress-test the readout.

    “Every geo test has a minimum detectable lift: the smallest effect it can reliably tell apart from noise. That threshold implies a budget floor.”

    Read it on workmagic.io

  • Phil KielOct 15, 2026Supports

    In one UK brand's Haus geo test, a Meta campaign to lapsed customers returned 1.93 incremental ROAS on returning-customer sales against Meta's reported 6.2x; Haus counted 40% of the orders Meta claimed.

    “a retention campaign showing 6x in Ads Manager is not a 6x campaign.”

    Read it on x.com

  • GoogleSep 10, 2026Caveat

    Google says its free, open-source Meridian GeoX library for geo experiments on any ad platform is now generally available, and its results can feed into a mix model.

    “Now, GeoX is generally available globally in Meridian, so you can get started with running independent experiments, or incorporating incrementality results into your MMM to help boost accuracy.”

    Read it on blog.google

  • LebesgueSep 7, 2026Caveat

    In one store's test, Lebesgue spent $6,695 on Meta ads in the UAE over about six weeks and estimates $7.7K to $9.7K of incremental revenue against two unadvertised markets.

    “This wasn't a perfect scientific holdout.”

    Read it on lebesgue.io

  • SellforteSep 3, 2026Supports

    Sellforte, which sells testing, says no fixed spend makes a channel too small to test; it is too small when no feasible design can detect an effect that would change the budget call. Then use broader evidence.

    “A full pause in selected units produces more signal than a shallow cut everywhere”

    Read it on sellforte.com

  • GoogleSep 3, 2026Supports

    Google's GeoX setup needs daily pre-test history by geo of at least three times the test length, and a year or more where seasonality is strong.

    “you'll provide a minimum of 3*N days worth of response data, such as revenue or conversion counts, where N is equals to the duration of the actual test period of your experiment”

    Read it on developers.google.com

  • GoogleSep 1, 2026Supports

    Google's GeoX guide calls for at least 10 geos for a single-cell test, ideally 50 to 100+, daily data only, and says geo tests usually cost more and run longer than user-based lift tests.

    “You should include a minimum of 10 geos for a single-cell design”

    Read it on developers.google.com

  • SellforteAug 27, 2026Supports

    Sellforte says geo tests often need 30% to 50% of eligible regions held out, and screens tests by wanting expected lift at least twice the detectable lift. Its examples are hypothetical.

    “In many practical designs, holding out roughly 30% to 50% of eligible geographies creates a stronger comparison.”

    Read it on sellforte.com

  • RecastAug 24, 2026Caveat

    Recast, which sells geo testing, advises geo tests of no less than two weeks, with a month as a good start, and says that for a fixed spend a shorter test is more powerful.

    “Counterintuitively, for a fixed amount of spend, a shorter test will be more powerful than a long one”

    Read it on geolift-docs.getrecast.com

  • RidgeAug 14, 2026Disagrees

    Ridge's Connor MacDonald says it mostly runs two and three week tests on Meta with about a one week post-treatment window, and is starting a six week test on upper-funnel channels.

    “we mostly run two and three week tests”

    Read it on podcasters.spotify.com

  • DarkroomAug 7, 2026Supports

    Darkroom, an agency, designs geo tests to detect a 5% lift with 12 to 15 markets per side, sized on your own history, and says too few markets can't be fixed by running longer. No results given.

    “eight markets selected by media availability can't be fixed with a longer runtime”

    Read it on darkroomagency.com

  • MeasuredJul 30, 2026Supports

    Measured, which sells geo testing, says incrementality tests commonly run four to eight weeks and almost never under four, since ending early misses lagged sales. Its illustrative Meta example needs about six weeks.

    “almost no test should run under four weeks regardless of what the power math allows”

    Read it on measured.com

  • MeasuredJul 30, 2026Supports

    Measured, which sells geo testing, says set the target lift from break-even economics. Detecting a 10% lift takes about 3 to 4 weeks at 20,000 weekly conversions, 12 or more at 1,000.

    “Set your target lift from economics (typically breakeven lift, not the lift you hope for)”

    Read it on measured.com

  • FlightedJul 16, 2026Supports

    Flighted, an agency, advises holding out 10 to 20% of the audience for 2 to 4 weeks or more, on campaigns spending $50K+ a month. Accounts under 100 to 200 conversions a week often struggle to reach significance.

    “often struggle to reach significance”

    Read it on flighted.co

  • Alex NeimanJun 17, 2026Supports

    Alex Neiman says freeze the account during a Conversion Lift test, budget two to three weeks since Haus's experiments averaged about 18.6 days, and test one focused question. He offers no data of his own.

    “Freeze the account. Don't restructure, change budgets, or launch new creative mid-test.”

    Read it on alexneiman.com

  • EightxJun 1, 2026Supports

    Eightx says geo holdouts should run 4 to 8 weeks, ideally 6, since 2-week tests usually fail, and are generally not worth it under $5M revenue. It shows no test data of its own.

    “Most first tests fail because the window is too short to clear noise.”

    Read it on eightx.co

  • ElevarusMay 30, 2026Supports

    Sets a lower floor than Sellforte's 30% to 50%: Elevarus says a holdout under 15% of regions is useless, and below $40K a month on a channel, use platform lift tests or skip testing.

    “A 5% holdout on a national account produces a control group too small to compare against anything.”

    Read it on elevarus.com

  • AmsiveMay 27, 2026Supports

    In a real client's YouTube geo test design (about $12.4K a month, 17 weeks), 4 control cities could detect a 21.5% lift and 10 cities 15.0%, so a true 10% lift would be missed.

    “If your team would reallocate budget at a +10% lift but not at a +5% lift, the test needs to distinguish those two outcomes.”

    Read it on amsive.com

  • Lewis and Rao ran 25 field experiments on Yahoo display ads. The median confidence interval on ROI was over 100 percentage points wide, so informative ad experiments can need over 10 million person-weeks.

    “experiments can easily require more than 10 million person-weeks”

    Read it on doi.org

In the full answer

For readers who sign in
  • What to do this week3 steps
  • The numbers behind it3 figures from cited sources
  • Where each source standsA chart placing each source on the question
  • The strongest disagreementThe best case against the answer, from a cited source
  • How sure we areWhat its confidence rests on, and what would change the answer

Read next

More in Measurement