How should we judge a creative test: what volume, what time window and which metric?
Our answer
The answer depends on spend.
Judge a new ad first by whether Meta spends on it at your bid. Then confirm on cost per purchase or ROAS over at least 7 days with enough purchases, not on click-through rate or hook rate.
At $200K to $1M a month, check loser spend. If more than 20% goes to ads switched off within 28 days of launch, you are cutting losers too slowly.
Why this is the answer
4 reasons and our judgmentAn ad's past ROAS doesn't predict its future ROAS (Common Thread Collective). The same agency found that ad-level stop-loss rules had no measurable effect on account results.
Small samples mislead. In 65,000 partnership ads, 45% of eventual winners still looked like losers before $100 of spend (Agentio).
A cost read needs enough conversions behind it. Meta says to judge a cost per result goal on weekly averages, over a window with at least 50 to 100 conversions.
Hook rate doesn't track which ads Meta spends on. Across 17 US accounts, Biddyco found ranking ads by spend and by hook rate gave a correlation close to zero, and the highest-hook ads had no lower cost per purchase.
Our judgment
- Read Meta's spend first, then confirm on results. We combined the spend signal from Charley Tichenor and the spend-based winner definition from Motion with the 7-day cost per purchase test from Opascope and Meta's conversion guidance for cost per result goals.
- Budget about 50 purchases in Meta's creative testing tool. We applied the low end of the conversions Meta gives for cost per result goals to that tool.
- Treat an ad that wins on spend inside an ad set like a winner in Meta's testing tool. This extends the work of Braun and Schwartz and of Burtch and colleagues, who studied Meta's A/B tests.
- Judge sale-period tests separately. This draws on Motion's note that its benchmark window spans Black Friday and Cyber Monday.
Our judgment, built from Charley Tichenor, Motion, Opascope, Meta and Michael Braun: our reading of these sources, not a finding from any one of them.
What the sources say
53 statements from 37 sourcesMeta says to judge cost and ROAS goals over a window with at least 50-100 conversions, using weekly averages, and to wait 7 days after a change before judging it.
Charley Tichenor puts about a 20% spend minimum on a test ad set that is not getting spend, then asks whether it let the campaign's budget go up. No results shared.
“Put a spend minimum on that ad set, starting around 20%. Then ask one question: did it make the campaign good enough to raise the budget?”
Mariano Petino-Paul says Meta has several times put spend behind new ads that were very engaging but did not sell by the brand's own metrics.
“very engaging, but that won't sell based on the metrics we recognize”
Marin Istvanic says how many days to look at in Ads Manager depends on AOV and conversion lag, and he usually finds 3 to 7 days the sweet spot.
“I usually find 3-7 days is a sweet spot”
Northbeam, which sells attribution, says Meta click-through rates are up more than 40% year over year across its 1,000+ accounts and about $6 billion in spend, and calls CTR partly a vanity metric.
“CTRs are, to some degree, a vanity metric.”
Cody Wittick's agency picks which ads to iterate on from 14-day results: 7 days is mostly noise while attribution fills in, and 21 to 30 days hides decline. Agency practice, no account data.
“7 days is too short. A new ad's first week is mostly noise.”
Marin Istvanic says an ad that keeps delivering after several budget raises in a week is ready for the cost cap scaling campaign; it only has to beat your target. Rule of thumb, no data.
“it just needs to be better than your target. You don't need statistical significance to realize that!”
Barry Hott says hook and hold rates mislead on optimized ads because they ignore audience quality and cost. He prefers to read the cost of each video play and each ThruPlay instead. Opinion, no data.
“Rates are misleading when talking about optimized ads.”
Northbeam's Bryan Bumgardner says to judge ads over at least your purchase lag. A 7-day window when buyers take 14 days uses half the signal. His $500K Black Friday loss claim has no data behind it.
“If you turn off ads BEFORE that length of time has fully passed, you're losing signal on which ads are actually working.”
CTC says across about 1.3 million ads it analyzed, the top 5% produced 73% of measured value, so winners should be judged net of the cost of ads that did not pay back. Method not stated.
“Across roughly 1.3 million ads we analyzed, the top 5% generated 73% of the measured value.”
An operator says to launch 10 concepts differing in format, persona or awareness level, with one CBO per product and one ad set per concept. One ad went from $500 to $12K+ in spend in 3 days. No account figures.
“A winner is the ad that becomes a top spender, so judge it by spend before anything else.”
Barry Hott says 80+% of marketers still get it wrong: don't rank or sort ads by CPA or ROAS when Meta is optimizing for purchases; focus on what gets the most spend. The 80% is his estimate.
“Don't rank or sort ads by CPA or ROAS.”
Ben Heath says to judge Meta campaigns by how many conversions they have had, not by how long they have run, and to be patient.
“This is all based on conversion volume. If you can be patient and keep going, that is super valuable.”
Phil Kiel says that on about $5K a month you should test 3 to 5 ads per ad set, not force each ad to 2,000 to 3,000 impressions, and move an ad you think deserves spend into its own ad set.
“Test 3 to 5 ads per ad set. 10 is too many at $5k a month.”
Pilothouse recommends opting out of Meta's product browsing before October 28 to keep the current ad experience through BFCM, warning that reported conversions may rise while AOV or margin fall. No test data.
“Duplicating an ad can turn the setting back on, so check every copy before it goes live.”
Meta says A/B tests and lift tests cost nothing extra. A/B tests compare up to 5 versions with no overlap, Conversion Lift needs a good data source such as Conversions API, and lift tests can change CPM.
“Lift tests can affect the overall cost per impression (CPM) of ad campaigns in your test depending on your test type and bidding strategy.”
Lorenzo Pravata says don't kill a still-spending ad over one bad CPA day or crown a winner at 4 ROAS on $50 of spend. One account grew from $29K to $150K a month on one CBO with about 95% of creative changed.
“Calling an ad a winner at a 4 ROAS on $50 of spend... Did it hold at $500?”
Meta's Insights API offers a creative_automation_asset_id breakdown to group results by the image or video delivered. Dynamic Creative asset breakdowns support only a few metrics, like impressions, clicks and spend.
“Use this breakdown to group results by the image or video that was delivered.”
An SSRN study using simulations and public Criteo logs finds more ad data can look more precise but be less reliable when users are counted repeatedly. Counting users cut precise Criteo campaigns from 75 to five.
“More advertising data can produce greater apparent precision under a mistaken independence assumption”
Meta told advertisers it would add in-app product browsing to existing ads from October 28, 2026 unless they opt out, claiming 3% more conversions in a test of about 117,700 advertisers. PPC Land could not verify it.
“Starting October 28, 2026, we'll add product browsing to your existing ads.”
Meta's creative test tool compares up to 7 copies of an existing ad inside the same campaign, so winners keep their learning. It suggests no more than 20% of budget and gives no confidence level.
“The test is set up in an existing campaign so that high-performing ads can continue to run after the test with delivery system learnings retained.”
David Herrmann says Meta is turning on product browsing for all existing ads from Oct 28, 2026, and on several brands an ad he thought was winning was mostly serving as a collection ad on Reels.
“This has now happened multiple times on multiple brands.”
Samuel Darby says to give each new ad at least 7 days, or let it spend twice your target cost per lead, before turning it off.
Eric Zaworski says to give a new ad about 5 days before judging it, not 48 hours, since an ad launched on a Friday reaches a different audience than one launched on a Monday.
Nick Theriot ranks offer over product page over creative, keeps 80% of effort on what is proven, gives tests 7 to 30 days, and says to model your own funnel numbers, not industry averages.
“all baselines / industry averages are bs, model your own funnel numbers”
Biddyco's study of 479 Meta video ads (17 US accounts, $10.1M) found spend and hook rate nearly uncorrelated. The highest-hook ads drew more clicks but paid higher CPMs and had no lower cost per purchase.
“They didn't have a lower cost per purchase.”
Ricardo Powell says that for complex, high-priced offers, top-of-funnel videos should be judged on engagement such as hook and hold rate, not last-click cost per acquisition.
Isaac Twidale says hook rate, click-through and video retention don't decide whether an ad makes money; use them to diagnose how to improve it.
“all of this stuff is not the determining factor of whether an ad is successful and people do get that wrong.”
Isaac Twidale says to judge an ad on spend, orders and cost per order against target, and use engagement metrics only to diagnose why it failed.
“Judge an ad based on how much was spent, how many orders you got... cost per order against your target... and then diagnose the ad with these metrics.”
Unnikrshnan Kurup says how many people viewed a video tells you nothing about whether it will turn into sales.
Megan Winter says on Meta Ads: Under the Hood that as a rule of thumb, about 2,000 impressions or 2,000 reach is enough to start analyzing a campaign test.
“if we've got about 2000 impressions or 2000 reach, depending on what we're doing, we can then analyze that”
Jetfuel relays benchmarks with no sample: AdSights puts the median 3-second hook rate near 23% for Meta feed cold traffic and 30% for Reels, while Taylor Sicard calls under 25% the kill zone.
“Notice the two sources disagree on where good starts, which is exactly why a single benchmark is dangerous.”
Agentio, which sells creator partnership ads, found in 65,000 Meta partnership ads ($130M spend) that 45% of eventual winners look like losers before $100 of spend; it suggests about $1,000 before cutting.
In CTC's study of cost controls across 253 accounts, ad-level stop-loss rules had no discernible effect on account-level performance.
Opascope, an agency, defines a winning ad as one holding CPA at or below target for 7 days after learning. It says across 200+ audited accounts, 5 to 7 percent of tested ads become winners. No method is given.
“Across 200+ audited accounts, 5 to 7 percent of tested ads turn into winners”
Wpromote's process runs new ads in a sandbox for at least 10 to 14 days against agreed ROAS thresholds, then graduates winners into evergreen campaigns. No data.
“Use Agreed-Upon KPI Thresholds To Determine Winning Ads”
CTC's Taylor Holiday says an ad's past ROAS does not predict its future ROAS, and Meta says the link between past ROAS and future delivery is not deterministic.
Motion says an ad that Meta spends on quickly is a better sign of a winning creative than an ad with high ROAS on small spend.
“high spend velocity, something interesting go go pay more attention to that than this one that just happens to have high ROAS but it's at small spend.”
Flighted judges creative tests by concept on 7-day click cost per purchase and sizes budgets for about 50 conversions in 7 days. It says results usually look worse the day after a change and settle by days 3 to 4.
“Use 7-day click attribution”
Jon Loomer says to judge new ads as a batch at the ad-set level: let them run an extra week after a creative test, then review again after a couple of weeks. No data.
“I don't care whether Meta's picking one ad over another.”
In one week of Wispr's video ads, the highest hook rate (39.5% watched 3 seconds) kept only 14.3% to the 25% mark and 3.6% to the end, and cost over $30 per download, against a winner about 40% cheaper.
“Wispr's highest hook rate of one week (39.5% watched 3 seconds) kept only 14.3% to the 25% mark and 3.6% to the end”
Wispr scores video ads with a Banger Score: 30% hook, 30% hold, 25% CTR and 15% CPM efficiency, each against the median ad in the concept. A score of 50 is the median and 75 or more is a winner.
“We think 30% uh, towards hook, 30% towards hold, 25% click-through rate, and 15% CPM efficiency times by 50.”
Segwise, a tool vendor with no data, says to test a challenger against a winner with equal daily budgets and one concept per ad set, give each variant at least 7 days, and judge early on hook rate, not CPA.
“Judge early potential on hook rate or 3-second view rate rather than CPA”
Rich From Anywhere says you can tell whether a new ad will win within 3 days of running it.
“you'll know if that ad is going to win within 3 days.”
Andrew Faris says that in more than 99% of cases you should not turn an ad off, and that switching off low-performing ads is often a disastrous decision.
In a live Meta campaign, a working paper found Meta's algorithm shifted who saw an ad based on its text metadata (+2.27 points female share), while the visible creative changed nothing.
“A conventional two-arm test would attribute the entire delivery shift to the creative.”
Tori Rowe says click-through rate and CPM analysis did not show which creatives would scale.
Tori Rowe says cost per add to cart was the number one early indicator of which creative went on to scale.
“the number one indicator of a Creative that ended up scaling was cost per add car”
Shamanth Rao says judge creative tests on rolling 7 to 14 day data, not day-1 ROAS. Strict significance needs 30 to 50 conversions per variant, and aim for 2 to 3 conversions a day per ad set.
“based on rolling 7-14 day data, not Day 1 ROAS”
Motion counts a winner as an ad spending at least 10x the account's median ad and at least $500, which about 5% of ads reach. Losers are ads turned off within 28 days.
Motion's benchmark defines a winning ad as one whose spend is at least 10 times the account median and at least $500. It is a benchmarking definition, not a test verdict rule.
A study of 3,204 Meta Lift tests and 181,890 A/B tests found Lift tests balanced, while A/B tests deliver to different audiences by design. To test creative alone, keep every other setting identical.
“Lift tests show no meaningful audience imbalance, confirming their causal validity, while A/B tests show clear imbalance, as expected”
A Journal of Marketing paper says platform A/B tools show ads to different mixes of users, so a result reflects ad content plus algorithmic selection. The winning ad may simply have reached better users.
“reflects the combination of ad content and algorithmic selection of users”
In the full answer
For readers who sign in- What to do this week3 steps
- The numbers behind it3 figures from cited sources
- Where each source standsA chart placing each source on the question
- The strongest disagreementThe best case against the answer, from a cited source
- How sure we areWhat its confidence rests on, and what would change the answer