All answers

The Prevailing
Answers for
Business model:
Business model, pick up to two

Should we use AI tools to analyze marketing data?

The PrevailingPublished Oct 29, 2026

Our answer

Use AI tools to pull, summarize and explore your marketing data with read-only access. Give them your metric definitions, check their totals against store numbers and treat their explanations as hypotheses, not decisions.

Confidence
Medium
Strongest proof
1 controlled test
3 official rules, 1 dataset, 7 practitioners, 2 opinions or claims
Sources
14 sources
14 statements

New here? Take the 1-minute tour

Why this is the answer

3 reasons and our judgment

AI gets the numbers right more often than the reasons. In e-dialog's test of Claude on a Meta account, the numbers matched its reports and the trend reading was strong. But the AI blamed ad fatigue for a campaign that had simply ended.

AI tools slip most on metric definitions. Precisian (ecommerce data modeling firm) finds the arithmetic right but the definitions guessed. Windsor.ai's logs show about 1 in 11 AI data queries failing, with Claude and ChatGPT counting GA4 users differently.

The makers of AI tools themselves say to check the output. Google says its analytics assistant may make mistakes, and Shopify says AI outputs can contain errors. In Spotify Engineering's test, raw AI predictions recovered only 39% of real headline test effects.

Our judgment

  • Treat any cause or recommendation as a hypothesis. Use AI tools to pull, summarize and explore data, and check what they say about why, because tests found the numbers usually right and the explanations sometimes wrong.
  • Give the assistant written metric definitions. Use one assistant for each recurring report, because assistants pick their own definitions and two of them count GA4 users differently.
  • Connect analysis tools with read-only access.
  • Don't let AI output stand in for a test. Test before moving budget.

Our judgment, built from e-dialog, PageFly, Common Thread Collective, Precisian, Windsor.ai, Meta, Shopify and Spotify Engineering: our reading of these sources, not a finding from any one of them.

What the sources say

14 statements from 14 sources
  • Marketing Operators podcastNov 10, 2026Supports

    On Marketing Operators, HexClad's Connor Rolain says his team turned a monthly report that took three weeks into a live AI dashboard. No performance figures given.

    Read it on podcasts.apple.com

  • ShopifyOct 29, 2026Supports

    Shopify says its Sidekick assistant drafts store copy and theme blocks, many AI features cost nothing extra, availability varies by plan, and AI output can contain errors that need review.

    “AI outputs can contain errors, so review anything a tool creates or changes.”

    Read it on help.shopify.com

  • PrecisianOct 14, 2026Supports

    Precisian, which sells a data layer, says ChatGPT gets the arithmetic right but invents definitions like revenue or ROAS when they are not in the file. Its main check: tie totals back to Shopify or the books.

    “It will make up the definition, then compute the number correctly on top of it.”

    Read it on precisian.io

  • Google2026-10Supports

    Google's beta Ask Advisor is a Gemini assistant in Google Analytics meant to diagnose drops and spikes, using only data from that one property. Google warns it may make mistakes.

    “Generated responses related to Google Analytics data rely only on information from within that specific Google Analytics property.”

    Read it on support.google.com

  • Meta says its ads MCP rules can give an AI agent read access to performance data while blocking budget, campaign or creative changes, and advises starting read-only or approval-required and widening access over time.

    “Start an account in a read-only or approval-required posture and widen permissions programmatically as confidence grows”

    Read it on developers.facebook.com

  • AppbrewSep 14, 2026Caveat

    Appbrew, which sells a rival app builder, says Shopify Sidekick is strong at drafting copy and building Flow automations but unreliable for bulk edits, custom code and precise numbers, and rarely admits gaps.

    “It's unreliable for bulk edits, custom code, metafields, and precise numbers.”

    Read it on appbrew.com

  • Windsor.aiAug 27, 2026Supports

    Windsor.ai, which sells the connector it logged, says AI assistants made about 11% of marketing data queries in one week across 35,000+ teams, and about 1 in 11 AI queries failed to run, whatever the vendor.

    “Pick one assistant per workflow and standardize.”

    Read it on windsor.ai

  • Spotify EngineeringAug 13, 2026Supports

    Spotify engineers found raw LLM predictions recovered only 39% of the real effect in thousands of headline A/B tests. They say LLMs can screen out weak ideas but cannot stand in for human tests.

    “can be used to filter out weak ideas before they consume an experiment slot”

    Read it on engineering.atspotify.com

  • Matt BertulliJun 29, 2026Caveat

    Matt Bertulli built AI tools and dashboards that pull live ad account data, but says he lost days debugging and shipped tools that broke the next day.

    “the prototype is not the product.”

    Read it on operatorscontent.com

  • Common Thread CollectiveJun 16, 2026Supports

    CTC runs Claude with written playbook skills over an MCP connection to analyze client Meta accounts, and its media buyers act as editors who check the output and add client context.

    “they're an editor, they're a filter, they get to gut check.”

    Read it on youtube.com

  • e-dialogJun 12, 2026Supports

    e-dialog tested Claude on a Meta ad account: figures matched its reports and trend analysis was strong, but causal explanations were often wrong, and Meta's own assistant leaned toward bigger budgets.

    “Meta’s proposals follow the platform’s logic, which tends to favor increased budgets and greater automation.”

    Read it on e-dialog.group

  • PageFlyJun 8, 2026Supports

    PageFly, which sells a page builder, tested Shopify Sidekick on one small demo store: its analytics summary matched the dashboard, but it cannot redesign pages. Treat output as a first draft.

    “Treat its output as a fast first draft, not a final answer.”

    Read it on pagefly.io

  • DTC Newsletter / DTC PodcastJun 3, 2026Disagrees

    True Classic's CEO says it runs 100% of its Meta spend through Triple Whale's autonomous media buyer, Moby 2, on an episode with Triple Whale's COO. No results or price are disclosed.

    “True Classic now runs 100 percent of its Meta spend through an autonomous media buyer, Moby 2”

    Read it on podbay.fm

  • 1ClickReportMay 21, 2026Caveat

    1ClickReport, which sells the tool used, asked Claude and ChatGPT 30 analytics questions on live accounts, scored by its founder: Claude did better on attribution, ChatGPT on ad platform specifics.

    “without live data access, both models guess plausibly but cannot verify.”

    Read it on 1clickreport.com

In the full answer

For readers who sign in
  • What to do this week3 steps
  • The numbers behind it2 figures from cited sources
  • Where each source standsA chart placing each source on the question
  • The strongest disagreementThe best case against the answer, from a cited source
  • How sure we areWhat its confidence rests on, and what would change the answer

Read next

More in Reporting and dashboards