Capture, testing and team

Test only big changes your traffic can read, and finish tests before your peak. Rent store development until the work fills a full-time role.

9 answers · 1 new or changed since Nov 16 · Data as of Nov 22, 2026

Answers for any spend, own site only, general category, United States.

The short version

Our judgment, drawn from the 9 answers below.

Test only big changes, and skip A/B tests below about 1,000 sessions or 20 to 30 conversions a month.

From Running A/B tests on the store

Rank site ideas on likely profit impact, evidence, cost and time to result, and fix obvious bugs at once.

From Prioritizing site improvements

Finish store tests before your peak and freeze theme, app and checkout changes 10 to 14 days ahead.

From Testing changes during peak season

Rent developers until store work fills a full-time role, and keep the code and access in your name.

From Store build: in house, agency or freelancers, How many people should run the store

The 9 answers

Grouped by the decision you are making. Jump to a group:

Test and prioritize

Which site ideas come first and how to run tests, including at peak.

  1. How should we prioritize the list of site improvements?

    Rank every site idea on likely profit impact, evidence, cost and time to result. Fix obvious bugs at once, favor offer changes over cosmetic ones, schedule big changes for quiet months and re-rank each quarter.

    Medium confidence · 7 sources

  2. How should a store our size run A/B tests, and how much traffic does a test need?

    Test only big changes. Plan for a few hundred orders in each version, run at least 2 full weeks and judge on revenue per visitor. Skip A/B tests below about 1,000 sessions or 20 to 30 conversions a month.

    Medium confidence · 10 sources

  3. How should we test changes during peak season?

    Run and call your store tests before your peak. Stop theme, app and checkout changes 10 to 14 days before it, and start no new A/B tests during it, because peak results rarely hold afterward.

    Medium confidence · 12 sources

Capture and listen

Sign-up pop-ups, review requests, chat assistants and turning customer questions into fixes.

  1. Should the site use sign-up pop-ups, and which offer and timing work best?

    Use a sign-up pop-up, but open it after a short delay and a scroll rather than on arrival. Show it at most twice a week, ask for email before phone and test it against no pop-up.

    Updated Nov 18 Medium confidence · 10 sources

  2. How should we gather customer reviews after purchase, and when should we ask?

    Ask every buyer once the product has arrived and been tried, 1 to 9 days after shipping for most products and later for slow-acting ones. Send one or two reminders, and reward photo or video reviews whatever their rating.

    Medium confidence · 13 sources

  3. Should we add live chat or an AI shopping assistant to the site?

    Add an AI chat assistant if shoppers ask questions before buying. Feed it accurate product and policy content, label it as AI and pass order problems and requests for a person to staff. Keep it if revenue per visitor holds.

    Medium confidence · 6 sources

  4. How should customer service questions feed back into site changes?

    Tag each ticket by reason and product and review the top themes weekly. Answer each repeated question on the page where it arises, then check that its tickets fall.

    Medium confidence · 7 sources

Staff the store

Who runs the store at your revenue and when to rent or hire.

  1. How many people should work on the store at our revenue, and in which roles?

    Below about $5M in yearly revenue, one owner runs the store and rents developers. From $5M to $7M, add a hybrid ecommerce manager who builds in Shopify. Add a dedicated developer once work is steady, often above $8M to $10M.

    Medium confidence · 5 sources

  2. Should we run the store in-house, with a Shopify agency or with freelancers?

    Use a vetted freelancer for theme-level store work and an agency for migrations or builds that span design and development. Hire in-house only once there is steady full-time work. Keep the code and access in your name.

    Medium confidence · 7 sources

More in Store

91 answers in 9 subjects

Guides that use these answers: How to plan for Black Friday and peak season · How to start email and SMS marketing · How to launch a Shopify store

See all 91 Store answers in one list