All answers

The Prevailing
Answers for
Business model:
Business model, pick up to two

How should we judge a creative test: what volume, what time window and which metric?

The PrevailingUpdated Nov 17, 2026

Our answer

The answer depends on spend.

Judge a new ad first by whether Meta spends on it at your bid. Then confirm on cost per purchase or ROAS over at least 7 days with enough purchases, not on click-through rate or hook rate.

At $200K to $1M a month, check loser spend. If more than 20% goes to ads switched off within 28 days of launch, you are cutting losers too slowly.

Why this is the answer

4 reasons and our judgment

An ad's past ROAS doesn't predict its future ROAS (Common Thread Collective). The same agency found that ad-level stop-loss rules had no measurable effect on account results.

Small samples mislead. In 65,000 partnership ads, 45% of eventual winners still looked like losers before $100 of spend (Agentio).

A cost read needs enough conversions behind it. Meta says to judge a cost per result goal on weekly averages, over a window with at least 50 to 100 conversions.

Hook rate doesn't track which ads Meta spends on. Across 17 US accounts, Biddyco found ranking ads by spend and by hook rate gave a correlation close to zero, and the highest-hook ads had no lower cost per purchase.

Our judgment

  • Read Meta's spend first, then confirm on results. We combined the spend signal from Charley Tichenor and the spend-based winner definition from Motion with the 7-day cost per purchase test from Opascope and Meta's conversion guidance for cost per result goals.
  • Budget about 50 purchases in Meta's creative testing tool. We applied the low end of the conversions Meta gives for cost per result goals to that tool.
  • Treat an ad that wins on spend inside an ad set like a winner in Meta's testing tool. This extends the work of Braun and Schwartz and of Burtch and colleagues, who studied Meta's A/B tests.
  • Judge sale-period tests separately. This draws on Motion's note that its benchmark window spans Black Friday and Cyber Monday.

Our judgment, built from Charley Tichenor, Motion, Opascope, Meta and Michael Braun: our reading of these sources, not a finding from any one of them.

What the sources say

53 statements from 37 sources
  • MetaUndatedSupports

    Meta says to judge cost and ROAS goals over a window with at least 50-100 conversions, using weekly averages, and to wait 7 days after a change before judging it.

    Read it on facebook.com

  • Charley TichenorNov 13, 2026Supports

    Charley Tichenor puts about a 20% spend minimum on a test ad set that is not getting spend, then asks whether it let the campaign's budget go up. No results shared.

    “Put a spend minimum on that ad set, starting around 20%. Then ask one question: did it make the campaign good enough to raise the budget?”

    Read it on x.com

  • Andrew YouderianNov 13, 2026Disagrees

    Mariano Petino-Paul says Meta has several times put spend behind new ads that were very engaging but did not sell by the brand's own metrics.

    “very engaging, but that won't sell based on the metrics we recognize”

    Read it on ecommercefuel.libsyn.com

  • Marin IstvanicNov 10, 2026Disagrees

    Marin Istvanic says how many days to look at in Ads Manager depends on AOV and conversion lag, and he usually finds 3 to 7 days the sweet spot.

    “I usually find 3-7 days is a sweet spot”

    Read it on x.com

  • Bryan BumgardnerNov 9, 2026Caveat

    Northbeam, which sells attribution, says Meta click-through rates are up more than 40% year over year across its 1,000+ accounts and about $6 billion in spend, and calls CTR partly a vanity metric.

    “CTRs are, to some degree, a vanity metric.”

    Read it on x.com

  • Cody WittickNov 6, 2026Caveat

    Cody Wittick's agency picks which ads to iterate on from 14-day results: 7 days is mostly noise while attribution fills in, and 21 to 30 days hides decline. Agency practice, no account data.

    “7 days is too short. A new ad's first week is mostly noise.”

    Read it on x.com

  • Marin IstvanicNov 6, 2026Supports

    Marin Istvanic says an ad that keeps delivering after several budget raises in a week is ready for the cost cap scaling campaign; it only has to beat your target. Rule of thumb, no data.

    “it just needs to be better than your target. You don't need statistical significance to realize that!”

    Read it on x.com

  • Barry HottNov 4, 2026Supports

    Barry Hott says hook and hold rates mislead on optimized ads because they ignore audience quality and cost. He prefers to read the cost of each video play and each ThruPlay instead. Opinion, no data.

    “Rates are misleading when talking about optimized ads.”

    Read it on x.com

  • Bryan BumgardnerNov 2, 2026Caveat

    Northbeam's Bryan Bumgardner says to judge ads over at least your purchase lag. A 7-day window when buyers take 14 days uses half the signal. His $500K Black Friday loss claim has no data behind it.

    “If you turn off ads BEFORE that length of time has fully passed, you're losing signal on which ads are actually working.”

    Read it on x.com

  • Common Thread CollectiveOct 23, 2026Supports

    CTC says across about 1.3 million ads it analyzed, the top 5% produced 73% of measured value, so winners should be judged net of the cost of ads that did not pay back. Method not stated.

    “Across roughly 1.3 million ads we analyzed, the top 5% generated 73% of the measured value.”

    Read it on x.com

  • Lorenzo PravataOct 21, 2026Supports

    An operator says to launch 10 concepts differing in format, persona or awareness level, with one CBO per product and one ad set per concept. One ad went from $500 to $12K+ in spend in 3 days. No account figures.

    “A winner is the ad that becomes a top spender, so judge it by spend before anything else.”

    Read it on x.com

  • Barry HottOct 20, 2026Supports

    Barry Hott says 80+% of marketers still get it wrong: don't rank or sort ads by CPA or ROAS when Meta is optimizing for purchases; focus on what gets the most spend. The 80% is his estimate.

    “Don't rank or sort ads by CPA or ROAS.”

    Read it on x.com

  • Ben HeathOct 20, 2026Supports

    Ben Heath says to judge Meta campaigns by how many conversions they have had, not by how long they have run, and to be patient.

    “This is all based on conversion volume. If you can be patient and keep going, that is super valuable.”

    Read it on youtube.com

  • Phil KielOct 20, 2026Supports

    Phil Kiel says that on about $5K a month you should test 3 to 5 ads per ad set, not force each ad to 2,000 to 3,000 impressions, and move an ad you think deserves spend into its own ad set.

    “Test 3 to 5 ads per ad set. 10 is too many at $5k a month.”

    Read it on x.com

  • Pilothouse recommends opting out of Meta's product browsing before October 28 to keep the current ad experience through BFCM, warning that reported conversions may rise while AOV or margin fall. No test data.

    “Duplicating an ad can turn the setting back on, so check every copy before it goes live.”

    Read it on directtoconsumer.co

  • MetaOct 17, 2026Caveat

    Meta says A/B tests and lift tests cost nothing extra. A/B tests compare up to 5 versions with no overlap, Conversion Lift needs a good data source such as Conversions API, and lift tests can change CPM.

    “Lift tests can affect the overall cost per impression (CPM) of ad campaigns in your test depending on your test type and bidding strategy.”

    Read it on facebook.com

  • Lorenzo PravataOct 15, 2026Supports

    Lorenzo Pravata says don't kill a still-spending ad over one bad CPA day or crown a winner at 4 ROAS on $50 of spend. One account grew from $29K to $150K a month on one CBO with about 95% of creative changed.

    “Calling an ad a winner at a 4 ROAS on $50 of spend... Did it hold at $500?”

    Read it on x.com

  • MetaOct 15, 2026Caveat

    Meta's Insights API offers a creative_automation_asset_id breakdown to group results by the image or video delivered. Dynamic Creative asset breakdowns support only a few metrics, like impressions, clicks and spend.

    “Use this breakdown to group results by the image or video that was delivered.”

    Read it on developers.facebook.com

  • Yevgeniy GolotaOct 15, 2026Caveat

    An SSRN study using simulations and public Criteo logs finds more ad data can look more precise but be less reliable when users are counted repeatedly. Counting users cut precise Criteo campaigns from 75 to five.

    “More advertising data can produce greater apparent precision under a mistaken independence assumption”

    Read it on doi.org

  • PPC LandOct 15, 2026Caveat

    Meta told advertisers it would add in-app product browsing to existing ads from October 28, 2026 unless they opt out, claiming 3% more conversions in a test of about 117,700 advertisers. PPC Land could not verify it.

    “Starting October 28, 2026, we'll add product browsing to your existing ads.”

    Read it on ppc.land

  • MetaOct 15, 2026Supports

    Meta's creative test tool compares up to 7 copies of an existing ad inside the same campaign, so winners keep their learning. It suggests no more than 20% of budget and gives no confidence level.

    “The test is set up in an existing campaign so that high-performing ads can continue to run after the test with delivery system learnings retained.”

    Read it on facebook.com

  • David HerrmannOct 14, 2026Caveat

    David Herrmann says Meta is turning on product browsing for all existing ads from Oct 28, 2026, and on several brands an ad he thought was winning was mostly serving as a collection ad on Reels.

    “This has now happened multiple times on multiple brands.”

    Read it on x.com

  • Samuel DarbyOct 14, 2026Supports

    Samuel Darby says to give each new ad at least 7 days, or let it spend twice your target cost per lead, before turning it off.

    Read it on youtube.com

  • Eric ZaworskiOct 12, 2026Caveat

    Eric Zaworski says to give a new ad about 5 days before judging it, not 48 hours, since an ad launched on a Friday reaches a different audience than one launched on a Monday.

    Read it on x.com

  • Nick TheriotOct 10, 2026Supports

    Nick Theriot ranks offer over product page over creative, keeps 80% of effort on what is proven, gives tests 7 to 30 days, and says to model your own funnel numbers, not industry averages.

    “all baselines / industry averages are bs, model your own funnel numbers”

    Read it on x.com

  • BiddycoSep 27, 2026Supports

    Biddyco's study of 479 Meta video ads (17 US accounts, $10.1M) found spend and hook rate nearly uncorrelated. The highest-hook ads drew more clicks but paid higher CPMs and had no lower cost per purchase.

    “They didn't have a lower cost per purchase.”

    Read it on biddyco.com

  • Ricardo PowellSep 22, 2026Disagrees

    Ricardo Powell says that for complex, high-priced offers, top-of-funnel videos should be judged on engagement such as hook and hold rate, not last-click cost per acquisition.

    Read it on perpetualtraffic.com

  • Igor IvitskiySep 22, 2026Supports

    Isaac Twidale says hook rate, click-through and video retention don't decide whether an ad makes money; use them to diagnose how to improve it.

    “all of this stuff is not the determining factor of whether an ad is successful and people do get that wrong.”

    Read it on youtube.com

  • Igor IvitskiySep 22, 2026Supports

    Isaac Twidale says to judge an ad on spend, orders and cost per order against target, and use engagement metrics only to diagnose why it failed.

    “Judge an ad based on how much was spent, how many orders you got... cost per order against your target... and then diagnose the ad with these metrics.”

    Read it on youtube.com

  • Claus LauterSep 14, 2026Supports

    Unnikrshnan Kurup says how many people viewed a video tells you nothing about whether it will turn into sales.

    Read it on ecommercecoffeebreak.com

  • Megan WinterAug 24, 2026Disagrees

    Megan Winter says on Meta Ads: Under the Hood that as a rule of thumb, about 2,000 impressions or 2,000 reach is enough to start analyzing a campaign test.

    “if we've got about 2000 impressions or 2000 reach, depending on what we're doing, we can then analyze that”

    Read it on lumemarketing.com

  • Jetfuel AgencyAug 20, 2026Disagrees

    Jetfuel relays benchmarks with no sample: AdSights puts the median 3-second hook rate near 23% for Meta feed cold traffic and 30% for Reels, while Taylor Sicard calls under 25% the kill zone.

    “Notice the two sources disagree on where good starts, which is exactly why a single benchmark is dangerous.”

    Read it on jetfuel.agency

  • AgentioAug 12, 2026Disagrees

    Agentio, which sells creator partnership ads, found in 65,000 Meta partnership ads ($130M spend) that 45% of eventual winners look like losers before $100 of spend; it suggests about $1,000 before cutting.

    Read it on agentio.com

  • Common Thread CollectiveAug 6, 2026Supports

    In CTC's study of cost controls across 253 accounts, ad-level stop-loss rules had no discernible effect on account-level performance.

    Read it on commonthreadco.com

  • OpascopeJul 17, 2026Supports

    Opascope, an agency, defines a winning ad as one holding CPA at or below target for 7 days after learning. It says across 200+ audited accounts, 5 to 7 percent of tested ads become winners. No method is given.

    “Across 200+ audited accounts, 5 to 7 percent of tested ads turn into winners”

    Read it on opascope.com

  • WpromoteJul 17, 2026Disagrees

    Wpromote's process runs new ads in a sandbox for at least 10 to 14 days against agreed ROAS thresholds, then graduates winners into evergreen campaigns. No data.

    “Use Agreed-Upon KPI Thresholds To Determine Winning Ads”

    Read it on motionapp.com

  • Common Thread CollectiveJul 7, 2026Supports

    CTC's Taylor Holiday says an ad's past ROAS does not predict its future ROAS, and Meta says the link between past ROAS and future delivery is not deterministic.

    Read it on youtube.com

  • MotionJul 3, 2026Supports

    Motion says an ad that Meta spends on quickly is a better sign of a winning creative than an ad with high ROAS on small spend.

    “high spend velocity, something interesting go go pay more attention to that than this one that just happens to have high ROAS but it's at small spend.”

    Read it on youtube.com

  • FlightedJul 1, 2026Supports

    Flighted judges creative tests by concept on 7-day click cost per purchase and sizes budgets for about 50 conversions in 7 days. It says results usually look worse the day after a change and settle by days 3 to 4.

    “Use 7-day click attribution”

    Read it on flighted.co

  • Jon LoomerJun 29, 2026Caveat

    Jon Loomer says to judge new ads as a batch at the ad-set level: let them run an extra week after a creative test, then review again after a couple of weeks. No data.

    “I don't care whether Meta's picking one ad over another.”

    Read it on pubcast.jonloomer.com

  • Eric ZaworskiJun 25, 2026Supports

    In one week of Wispr's video ads, the highest hook rate (39.5% watched 3 seconds) kept only 14.3% to the 25% mark and 3.6% to the end, and cost over $30 per download, against a winner about 40% cheaper.

    “Wispr's highest hook rate of one week (39.5% watched 3 seconds) kept only 14.3% to the 25% mark and 3.6% to the end”

    Read it on motionapp.com

  • Eric ZaworskiJun 25, 2026Disagrees

    Wispr scores video ads with a Banger Score: 30% hook, 30% hold, 25% CTR and 15% CPM efficiency, each against the median ad in the concept. A score of 50 is the median and 75 or more is a winner.

    “We think 30% uh, towards hook, 30% towards hold, 25% click-through rate, and 15% CPM efficiency times by 50.”

    Read it on motionapp.com

  • SegwiseJun 22, 2026Disagrees

    Segwise, a tool vendor with no data, says to test a challenger against a winner with equal daily budgets and one concept per ad set, give each variant at least 7 days, and judge early on hook rate, not CPA.

    “Judge early potential on hook rate or 3-second view rate rather than CPA”

    Read it on segwise.ai

  • Rich From AnywhereJun 22, 2026Disagrees

    Rich From Anywhere says you can tell whether a new ad will win within 3 days of running it.

    “you'll know if that ad is going to win within 3 days.”

    Read it on youtube.com

  • Andrew FarisMay 27, 2026Supports

    Andrew Faris says that in more than 99% of cases you should not turn an ad off, and that switching off low-performing ads is often a disastrous decision.

    Read it on youtube.com

  • In a live Meta campaign, a working paper found Meta's algorithm shifted who saw an ad based on its text metadata (+2.27 points female share), while the visible creative changed nothing.

    “A conventional two-arm test would attribute the entire delivery shift to the creative.”

    Read it on econpapers.repec.org

  • Andrew FoxwellMay 21, 2026Supports

    Tori Rowe says click-through rate and CPM analysis did not show which creatives would scale.

    Read it on share.transistor.fm

  • Andrew FoxwellMay 21, 2026Caveat

    Tori Rowe says cost per add to cart was the number one early indicator of which creative went on to scale.

    “the number one indicator of a Creative that ended up scaling was cost per add car”

    Read it on share.transistor.fm

  • RocketShip HQMay 18, 2026Supports

    Shamanth Rao says judge creative tests on rolling 7 to 14 day data, not day-1 ROAS. Strict significance needs 30 to 50 conversions per variant, and aim for 2 to 3 conversions a day per ad set.

    “based on rolling 7-14 day data, not Day 1 ROAS”

    Read it on rocketshiphq.com

  • MotionApr 17, 2026Supports

    Motion counts a winner as an ad spending at least 10x the account's median ad and at least $500, which about 5% of ads reach. Losers are ads turned off within 28 days.

    Read it on motionapp.com

  • MotionApr 17, 2026Caveat

    Motion's benchmark defines a winning ad as one whose spend is at least 10 times the account median and at least $500. It is a benchmarking definition, not a test verdict rule.

    Read it on motionapp.com

  • MetaAug 28, 2025Supports

    A study of 3,204 Meta Lift tests and 181,890 A/B tests found Lift tests balanced, while A/B tests deliver to different audiences by design. To test creative alone, keep every other setting identical.

    “Lift tests show no meaningful audience imbalance, confirming their causal validity, while A/B tests show clear imbalance, as expected”

    Read it on arxiv.org

  • Michael BraunJan 7, 2025Supports

    A Journal of Marketing paper says platform A/B tools show ads to different mixes of users, so a result reflects ad content plus algorithmic selection. The winning ad may simply have reached better users.

    “reflects the combination of ad content and algorithmic selection of users”

    Read it on scholar.smu.edu

In the full answer

For readers who sign in
  • What to do this week3 steps
  • The numbers behind it3 figures from cited sources
  • Where each source standsA chart placing each source on the question
  • The strongest disagreementThe best case against the answer, from a cited source
  • How sure we areWhat its confidence rests on, and what would change the answer

Read next

More in Creative volume and testing