How do we test whether an SEO change worked?
Our answer
Change a random half of similar pages, leave the rest as a control and compare them for 2 to 8 weeks. With too few pages, compare clicks before and after against past periods, changing one thing at a time.
- Confidence
- Medium
- Strongest proof
- 3 controlled tests
- 3 official rules, 3 practitioners, 3 opinions or claims
- Sources
- 8 sources
- 12 statements
- SG
SSALNS
New here? Take the 1-minute tour
Why this is the answer
3 reasons and our judgmentA control group separates your change from seasonality and algorithm updates. SearchPilot and SEOTesting, which sell testing tools, compare changed pages with unchanged ones over the same weeks, and class plain before and after reads as weaker.
Splitting similar pages into changed and control groups needs scale. SearchPilot works with hundreds of template pages and 30,000 monthly organic sessions and sees results in 2 to 4 weeks. SEOTesting suggests 6 to 8 weeks for small and medium sites.
When you can't split pages, read before and after with care. Google says wait a few weeks to judge a change and compare its timing with similar past periods. Atticus Li says before and after with forecasting suits changes that cannot be split.
Our judgment
- A test that splits pages into changed and control groups runs about 2 to 8 weeks. We drew that range from SearchPilot's timing on high traffic templates and SEOTesting's for small and medium sites.
- Most small businesses won't reach the hundreds of template pages and 30,000 monthly organic sessions SearchPilot works with, so their usual test is a before and after read. Compare in Search Console against the same weeks a year earlier and the prior period, annotate the date, rule out other changes and read it as a weaker signal.
- Decide in advance what result counts as failure, and keep one change per test. Giulia Panozzo and SearchCounselCo say several changes at once hide which one worked, and SearchPilot's inconclusive alt text test shows a real range can cross zero.
- End any test that serves different page versions once it has an answer, and keep it within Google's rules. Google warns that long experiments may be read as deception, even though John Mueller says varying content brings no penalty.
Our judgment, built from SearchPilot, SEOTesting, Google, Atticus Li, Giulia Panozzo, SearchCounselCo and Search Engine Journal: our reading of these sources, not a finding from any one of them.
What the sources say
12 statements from 8 sourcesSearchPilot, which sells SEO testing, says SEO split tests randomly assign pages to control and variant groups, generally need hundreds of template pages and 30,000 organic sessions a month, and take 2 to 4 weeks.
“Pages should be randomly assigned as a control or variant page.”
Google advises watching trends in impressions and clicks more than average position, and checking that Search gains line up in time with your site changes, compared with similar past periods.
“In general, it's recommended to focus more on trends in impressions and clicks than on position alone.”
Giulia Panozzo, a consultant, warns that a metric moving after an SEO change is weak evidence; decide what would disprove the idea, change one thing at a time, and check for seasonality or updates.
“You can make a change, watch a metric move, and still have very little evidence that what you implemented caused it.”
SearchCounselCo, an SEO consultancy, says a traffic drop during a core update makes it only a possible cause; track a fixed set of affected pages and make the smallest change the evidence supports.
“Do not rewrite an entire site because one traffic graph dropped during the rollout.”
In a seven-week seoClarity split test on 70 category pages of one tools retailer, adding unique copy above the product grid lifted organic clicks, significant at 99.8%. seoClarity sells the tool.
“The lift comes from content written for each category, not duplicated or strictly templatized across pages.”
Google's John Mueller said on Bluesky there is no penalty for long A/B tests, though very different versions may show in search. The article says this seems to contradict Google's written guidance.
“If they’re significantly different, that could be visible in search results too.”
SearchPilot's split test on one classifieds site found descriptive image alt text had no statistically significant effect on organic traffic; the estimate was +7.3%, with a range crossing zero.
“The results of this test were inconclusive. We did not observe any statistically significant change in organic traffic in either direction.”
In a SearchPilot test on one ecommerce site, adding page-specific FAQ content below listing page footer copy raised organic clicks an estimated 9.7%.
“The test delivered a positive result at the 95% credible interval, with an estimated +9.7% increase in organic clicks.”
Atticus Li, a CRO consultant, says before/after SEO comparisons are unreliable because of seasonality and algorithm changes; split pages into test and control groups, or use causal impact models for sitewide changes.
“It is also deeply unreliable.”
Google says SEO changes can take from a few hours to several months to show, so wait a few weeks before judging results, and not every change will have a noticeable effect.
“In general, you likely want to wait a few weeks to assess whether your work had beneficial effects in Google Search results.”
Google says page tests should point variant URLs to the original with canonical tags, use 302 not 301 redirects, never show Googlebot different URLs, and end promptly. Small changes rarely affect rankings.
“Don't show one set of URLs to Googlebot, and a different set to humans.”
SEOTesting, which sells SEO testing software, says to split same-template pages into control and test groups, which shows cause better than before-and-after tests, and to run tests 6 to 8 weeks.
“When you compare results over multiple pages with a control group, you can determine causation rather than just correlation.”
In the full answer
For readers who sign in- What to do this week3 steps
- The numbers behind it3 figures from cited sources
- Where each source standsA chart placing each source on the question
- The strongest disagreementThe best case against the answer, from a cited source
- How sure we areWhat its confidence rests on, and what would change the answer