When we try a new Meta setting, such as product browsing or profit bidding, how do we test it against our current setup?
Our answer
Test a new Meta setting as two matched cells at equal budget, about 100 purchases each over at least two weeks, ignoring week one. Decide on new-customer cost or margin in your own numbers, not Meta's.
- Confidence
- Medium
- Strongest proof
- 1 controlled test
- 4 Meta rules, 1 dataset, 4 practitioners, 9 opinions or claims
- Sources
- 13 sources
- 19 statements
- M
New here? Take the 1-minute tour
Why this is the answer
3 reasons and our judgmentMeta's A/B test splits the audience so nobody sees both versions, and Meta recommends equal budgets. Its help page warns that testing by switching campaigns on and off can give unreliable results.
Meta's numbers under two settings only show which setting claims more credit (Haus). In its experiments, Incremental Attribution earned 0.80 times standard attribution's incremental return from July 2024 to June 2025, and 1.26 times a year later.
A decision needs about 100 purchases per cell and a directional read about 50 (Tracify). In Brad Ploch's test at $150K to $200K a month, the new setting looked terrible in week one and led in weeks two and three.
Our judgment
- We set about 100 purchases per cell for a decision and about 50 for a directional read, over at least two weeks without judging on week one. That joins Tracify's two levels and its 2 to 3 week typical run with Brad Ploch's test. So we read Common Thread Collective's 7 to 10 day product browsing test as directional, since it is shorter than Tracify's typical run. WeltPixel's sample-size arithmetic, applied here from holdouts to two even cells, puts 100 per cell at about a 50% difference in purchases, not the 20% in cost per purchase Tracify gives.
- Judge on new-customer cost or margin in your own numbers. This combines Haus (compare incremental return), Tracify (first-party new-customer cost against a threshold set in advance), Common Thread Collective (first-party conversion data), Ads Uploader (blended revenue) and TrueROAS (month-two profit), with the margin target in The business metric that governs spend.
- Pick the test by the question: incremental sales, or which setup Meta delivers better. Use a conversion lift test with a holdout for each cell for incremental sales, and Meta's A/B test for which setup Meta delivers better. This draws on Haus, Ploch's holdout per cell and the 2025 paper showing lift tests stay balanced while A/B tests do not.
- Switch over only when the setting can't be split into cells. Meta advises against informal on-and-off tests. Tracify calls its before-and-after option directional, and TrueROAS reads month two.
- Retest a winning setting after Meta changes it. Haus's Incremental Attribution result reversed between years. Ploch's winner drifted after Meta's March 2026 click change, so he plans a retest in 3 to 6 months.
Our judgment, built from Tracify, Brad Ploch (Scalability School), Haus, Common Thread Collective, Ads Uploader, TrueROAS, Meta and WeltPixel: our reading of these sources, not a finding from any one of them.
What the sources say
19 statements from 13 sourcesMeta's Conversion Lift has no extra cost. As a guide, the account needs a campaign from the past year with $5,000+ spend and 500+ conversions, plus Conversions API with event match quality above 5.
“There’s no additional cost for the test, but ad campaigns or sets you use for conversion lift tests may have certain budget requirements.”
PPC Land relays Meta's advice to check the largest campaign's optimization target and then hold it still, grouping changes so one learning phase replaces several. No data on how often edits occur or what they cost.
“hold it still”
A research paper proposes group sequential A/B tests that allow peeking without invalid results, cutting expected duration 43.1% (7.97 days against 14). It was tested on online experiments, not ad tests.
“reduces expected duration by 43.1% (7.97 days vs. 14-day baseline)”
Meta reports that agency Tinuiti tested one campaign fed with DSW's in-store sales against the old setup: 23% more purchases, 45% more in stores and 15% better ROAS. No spend or duration is given.
“a strong month on its own would not have established that the new structure caused the improvement.”
Meta suggests starting with an account level Conversion Lift or Brand Lift test, then A/B tests, then per-campaign or combined lift and A/B tests. Some lift tests have spend minimums.
“If you're new to testing and experimenting, you can start with an account level test to measure the value of your Meta advertising independent of all your other marketing efforts”
Meta's A/B testing help page says to use the same budget for both versions, and advises against informal testing by manually turning ad sets or campaigns on and off.
“We recommend using the same budget for both versions in a test to ensure a fair comparison.”
Meta says A/B tests and lift tests cost nothing extra. A/B tests compare up to 5 versions with no overlap, Conversion Lift needs a good data source such as Conversions API, and lift tests can change CPM.
“Lift tests can affect the overall cost per impression (CPM) of ad campaigns in your test depending on your test type and bidding strategy.”
CTC relays Meta's claim of 3% more conversions with product browsing in a 15-day test across about 117,700 advertisers, warns it may not hold for every brand, and suggests a 7 to 10 day on/off test.
“Test one ad with product browsing on against one without for 7 to 10 days before Black Friday”
CTC suggests testing one ad with Meta's product browsing on against one without for 7 to 10 days before Black Friday, judged on first-party data, after auditing the catalog. Suggestions may show competitors.
“Test one ad with product browsing on against one without for 7 to 10 days”
Ads Uploader, which sells an ad launching tool, reports Meta will switch on product browsing for running ads around October 28, opt-out only. It advises opting out, then testing it in one campaign.
“It is opt-out only, and Meta hasn't documented it publicly yet.”
CustomerLabs, which sells the integration, sends order profit to Meta for value bidding, but says the feature must be enabled in your account, old ROAS targets should not carry over, and gains are not promised.
“Never copy a legacy ROAS target straight into a profit-based campaign”
Involve Digital, an agency, says B2B targeting on Meta works best from customer data you hold, since job titles are self-declared. Test Advantage+ against manual prospecting for about four weeks.
“Test Advantage+ against your best manual prospecting at the same budget for about four weeks, judge the two on cost per qualified lead, and keep retargeting on audiences you define.”
WeltPixel, a Shopify tracking app maker, works out that a holdout needs about 475 purchases per arm to detect a 20% lift and about 1,730 for 10%. This is its own arithmetic, not a test.
“If that arithmetic says nine weeks, the test is nine weeks.”
Elevarus, an agency with no data, says to settle Meta's Advantage+ gains against Haus's incrementality results by running automated and manual builds against each other in a geo split or randomized holdout.
“Run a holdout rather than a before-and-after comparison.”
TrueROAS, a vendor, says to collect profit values as a secondary conversion for about 30 days before making them primary and judge on month-two profit, not week-one ROAS. Starting POAS is 1.2 to 1.7 (no controlled test).
“Judge the switch on month-two profit, not week-one ROAS”
Haus's incrementality tests found Meta's Incremental Attribution returned 0.80x the incremental return per dollar of standard attribution in July 2024 to June 2025, rising to 1.26x in July 2025 to June 2026.
“IA iROAS is beating standard iROAS by a pooled geo-mean of 1.26x”
Brad Ploch says for one apparel brand at $150-200K a month, Incremental Attribution passed a lift test but drifted after Meta's March click change, so they went back to one-day click and plan to retest.
Tracify, which sells tracking, proposes testing a new data source with Meta's Experiments A/B test, using about 100 conversions per campaign for a business decision and about 50 for a directional read.
“cleanest methodology”
A study of 3,204 Meta Lift tests and 181,890 A/B tests found Lift tests balanced, while A/B tests deliver to different audiences by design. To test creative alone, keep every other setting identical.
“Lift tests show no meaningful audience imbalance, confirming their causal validity, while A/B tests show clear imbalance, as expected”
In the full answer
For readers who sign in- What to do this week3 steps
- The numbers behind it3 figures from cited sources
- Where each source standsA chart placing each source on the question
- The strongest disagreementThe best case against the answer, from a cited source
- How sure we areWhat its confidence rests on, and what would change the answer