AI engines give different answers because every answer is assembled twice — once when the engine decides what to read, and again when the model decides what to say about it. Both steps differ per engine, and they fail independently. That is why the same question produces three different shortlists in ChatGPT, Gemini and Perplexity without any of them being broken.
Most coverage of this treats it as one phenomenon. It isn't. We ran an identical prompt set through all eight major engines simultaneously and tagged every disagreement by its cause. The split matters, because a retrieval problem and a model problem need completely different fixes — and different budgets.
The short answer: retrieval and model behavior split the blame almost evenly
Cross-engine divergence has two causes. Retrieval divergence happens when an engine never sees the pages that would have named your brand. Model divergence happens when the engine sees them and still leaves you out. In our 3,600-answer sample, 54% of disagreements traced to retrieval and 46% to model behavior — close to a coin flip, not the index-dominated story usually told.
That near-even split is the single most useful thing to know here. It means "get cited on more sites" solves only about half your problem. The other half is how your evidence reads once an engine already has it.
How we ran the test: 150 prompts, 8 engines, 3 repeats
We built a fixed prompt set of 150 buyer-intent questions — 30 B2B software categories, five phrasings each ("best X tools", "what should I use for X", "X alternatives for a 200-person company", and so on). Every prompt ran through all eight engines: ChatGPT, Google Gemini, Perplexity, Claude, Microsoft Copilot, Grok, Google AI Mode and Google AI Overviews.
Three details make the results usable rather than anecdotal:
- Every prompt ran three times, on three consecutive days, so we could separate engine-vs-engine difference from an engine's disagreement with itself.
- Conditions were held flat: US locale, fresh logged-out sessions, no memory or personalization, default model tier, single turn only, all eight engines queried inside the same two-hour window.
- We captured four things per answer, not just brand names: the ordered brand list, every cited URL, the one-line descriptor applied to each brand, and the answer's structure.
Total corpus: 150 prompts × 8 engines × 3 runs = 3,600 answers, carrying 26,400 citations across 9,140 unique URLs.
Known limits of this sample. It is US-English, B2B software, logged-out, single-turn, default-tier. Personalized accounts with memory enabled behave differently, and so does every non-English market — AI recommends materially different brands in each language, so treat these numbers as a structure to replicate, not constants to inherit.
Result 1: Engines barely agree with themselves, let alone each other
Before blaming engines for disagreeing with each other, measure how much each one disagrees with itself. Running the same prompt through the same engine on three different days produced a mean brand-list overlap of just 0.62 (Jaccard). Across different engines on the same day, mean overlap was 0.29.
That ratio is the finding. Engines reach only about 47% of the agreement their own stability would even permit. Half the gap between engines isn't an engine difference at all — it's run-to-run noise that any single screenshot would have misattributed.
| Engine | Agreement with itself (3 runs) | Mean agreement with other 7 | Ratio |
|---|---|---|---|
| Perplexity | 0.74 | 0.33 | 0.45 |
| Google AI Overviews | 0.71 | 0.28 | 0.39 |
| Google AI Mode | 0.66 | 0.31 | 0.47 |
| Microsoft Copilot | 0.64 | 0.32 | 0.50 |
| Gemini | 0.62 | 0.30 | 0.48 |
| ChatGPT | 0.58 | 0.29 | 0.50 |
| Claude | 0.55 | 0.25 | 0.45 |
| Grok | 0.49 | 0.22 | 0.45 |
| Mean | 0.62 | 0.29 | 0.47 |
This lines up with — and extends — the SparkToro and Gumshoe study finding that AI recommendation lists almost never repeat exactly. They measured exact-list repetition across three engines and consumer categories. We measured set overlap across eight engines and B2B categories, which is a looser test — and even under the looser test, self-agreement tops out at 0.74. Perplexity, the most retrieval-anchored engine in the set, is the most stable. Grok, the most recency-driven, is the least.
Practical consequence: any AI search monitoring that reports a "ranking position" from a single run is reporting noise. Presence is a far more stable signal than position — across our three runs, whether a brand appeared at all changed in 22% of cases, while where it appeared changed in 61%.
Result 2: The Google natural experiment — same index, different answer
Google AI Mode and Google AI Overviews are the cleanest test available of whether model behavior matters independently of retrieval. Both are Google. Both draw on Google's index. If retrieval explained everything, they should agree almost perfectly.
They don't. In our set, the AI Mode ↔ AI Overviews pair was the highest-agreeing pair of all 28 engine pairs — and still only 0.39. Two systems inside the same company, pointed at the same index, given the same question, produce brand lists that overlap around two-fifths of the time.
That is directionally consistent with Ahrefs' analysis of 540,000 query pairs, which found only 13.7% citation overlap between AI Overviews and AI Mode despite 86% semantic similarity in the answers. Their measure and ours diverge in an instructive way: answers can read alike while naming different companies. Semantic similarity is generous; brand-set overlap is not. For a marketer, only the second one pays.
| Highest-agreeing pairs | Overlap | Lowest-agreeing pairs | Overlap |
|---|---|---|---|
| Google AI Mode ↔ AI Overviews | 0.39 | Grok ↔ AI Overviews | 0.14 |
| ChatGPT ↔ Microsoft Copilot | 0.37 | Claude ↔ Grok | 0.17 |
| Gemini ↔ Google AI Mode | 0.36 | Claude ↔ AI Overviews | 0.19 |
| Perplexity ↔ ChatGPT | 0.34 | Grok ↔ Gemini | 0.20 |
| Gemini ↔ AI Overviews | 0.33 | Claude ↔ Copilot | 0.22 |
The pattern in the left column is shared infrastructure — Google systems cluster, and ChatGPT and Copilot both lean on Bing-adjacent retrieval. The right column is where retrieval philosophies collide: Grok's recency bias against AI Overviews' conservatism produces near-total disagreement. Our wider study of how much the major engines overlap on brand picks tracks these pair scores over a longer window.
The divergence decomposition: four layers, four different fixes
We sampled 1,200 divergence events — cases where one engine recommended a brand and another, answering the same prompt on the same day, did not — and traced each to a cause by checking whether the brand's evidence appeared in the second engine's citation set.
Four layers accounted for all of it:
| Layer | Share | What actually happened | What fixes it |
|---|---|---|---|
| Retrieval | 54% | No page from the brand's citation footprint appeared anywhere in the engine's sources | Earn coverage on sources that engine reads |
| Selection | 23% | The pages were cited, but the brand still wasn't named | Make the claim extractable on the page |
| Ranking | 13% | The brand was mentioned, but below the recommended set | Strengthen comparative evidence |
| Format | 10% | The engine answered a different shape of question entirely | Match the answer format |
Layer 1 — Retrieval divergence (54%)
The most common failure is invisibility, not rejection. The engine never encountered a document that would have named you. This is usually an index-coverage problem: your evidence lives on sources one engine crawls often and another crawls rarely, or hasn't crawled since your last positioning change.
Query fan-out amplifies this. Google's documentation confirms both AI Overviews and AI Mode issue multiple related searches behind a single question to surface a wider set of links, and every engine runs some version of this. A brand that matches the literal prompt but not the hidden sub-queries drops out silently. Understanding how one prompt becomes dozens of hidden searches explains most "we rank but we're not cited" cases.
Layer 2 — Selection divergence (23%)
Here the engine did cite a page that mentions you and still left you out. Nearly a quarter of divergence sits in this gap. The cause is almost always structural: your brand appears as a passing mention, a logo, a table cell, or a sentence that needs three paragraphs of surrounding context to make sense.
Engines extract claims, not pages. If the sentence naming your brand doesn't also state what you do and who you're for, it survives retrieval and dies in selection. This layer is entirely within your control and is the cheapest to fix.
Layer 3 — Ranking divergence (13%)
The brand made the answer but not the recommendation. It appeared as an "also worth mentioning" trailer, a footnote, or a comparison foil. Because every engine caps how many brands it will actually endorse, being present is not the same as being picked — a dynamic covered in more depth in our analysis of how many brands an AI answer will actually recommend.
Mean brands recommended per answer varied by nearly 2x across engines:
| Engine | Mean brands recommended | Median age of cited source |
|---|---|---|
| Perplexity | 6.4 | 5 months |
| Grok | 5.8 | 2 months |
| Gemini | 5.2 | 11 months |
| ChatGPT | 4.9 | 9 months |
| Microsoft Copilot | 4.5 | 8 months |
| Google AI Mode | 4.1 | 7 months |
| Claude | 3.8 | 12 months |
| Google AI Overviews | 3.0 | 16 months |
Read those two columns together. Perplexity and Grok give you roughly twice the slots that AI Overviews does, and they read sources three to eight times fresher. A brand that earned strong coverage six months ago is competitive in Perplexity and effectively invisible in AI Overviews — not because AI Overviews dislikes it, but because AI Overviews is still reading last year's web.
Layer 4 — Format divergence (10%)
One in ten disagreements isn't about your brand at all. The engine returned product categories instead of company names, refused to rank and listed "factors to consider," asked a clarifying question, or returned two names when its peers returned six. Claude does this most often — it hedges toward criteria over verdicts — which also changes what earns a citation there; our notes on how Claude searches the web cover that behavior. Grok hedges least.
You cannot fix this by earning more citations. You fix it by making sure your evidence works whether the engine wants a list, a comparison, or a definition.
Where the eight engines actually get their sources
Retrieval divergence is only actionable once you know which pipeline feeds which engine. The eight engines draw on roughly four distinct retrieval backbones — Google's index, Bing's index, independent crawlers, and social/real-time streams — with several engines blending two. Our map of which search index powers each AI engine breaks down the current wiring.
The measurable consequence is citation fragmentation. Mean pairwise citation overlap across our eight engines was 12% — the same two engines answering the same question shared roughly one source in eight. Only 3.1% of cited URLs appeared in six or more of the eight engines.
Those rare pages are disproportionately valuable. Brands whose evidence appeared on at least one such consensus page were recommended by a median of 5 engines out of 8. Brands cited only by a single engine were recommended by a median of 2. That's the highest-use asymmetry in the whole dataset: consensus pages — the category roundups, review platforms and reference sites that every backbone crawls — deserve their own target list rather than being treated as ordinary link placements.
Result 3: Engines don't just pick different brands — they describe you differently
Divergence isn't only about presence. It's about wording. We pulled the one-line descriptor each engine applied to 40 tracked brands. All eight engines agreed on the basic category label for only 9 of 40 brands (23%). For the rest, at least one engine placed the company in a different category than its peers.
Worse, 17 of 40 brands (43%) had at least one engine attaching a stale attribute — a deprecated pricing tier, a pre-rebrand name, a discontinued feature, or an outdated funding fact. The offender was predictable from the table above: Google AI Overviews, with a median citation age of 16 months, carried stale attributes most often.
This is a distinct workstream from citation building, and it's the part most teams discover too late. The mechanics of why models describe the same brand differently are worth reading alongside this data, because reputation repair and citation acquisition need different tactics. The divergence also reaches beyond product questions — answers about what your company is like to work for fragment the same way, from a different source pool.
Does divergence actually cost you traffic?
Short answer: it costs you less traffic than it costs you consideration. Only a minority of AI answers produce a click at all, so being omitted from an answer mostly means being omitted from the buyer's mental shortlist — the damage shows up later as an absent vendor in an RFP, not as a dip in sessions. Our data on post-answer click behavior by engine and position breaks the click side down.
Two consequences follow:
- Don't measure AI visibility in analytics. Referral traffic from AI engines undercounts influence badly, and most of it arrives attributed to direct or branded search.
- Measure presence in the answer instead, per engine, across repeated runs. That is the variable that moves your pipeline.
What to actually do about it
Fix the layers in order of their share. Chasing selection problems while a retrieval problem is unsolved wastes the quarter.
- Close the retrieval gap first (54% of the problem). List the engines that never cite your footprint. For each, pull the domains it does cite in your category from 20–30 prompts, and rank them by how many of your competitors they name. Earn coverage on those, in that order — engine coverage is source-specific, so broad link acquisition underperforms a short targeted list.
- Make the naming sentence self-contained (23%). Every page that mentions you should have one sentence stating what you are, who you're for, and what you replace — readable without surrounding context. Cheapest win available, and it lifts all eight engines at once.
- Give ranking layers something to compare (13%). Engines demote brands they can describe but not differentiate. Specific, verifiable comparison points — pricing model, integration depth, deployment type — move brands from the "also mentioned" trailer into the recommended set.
- Cover all three answer shapes (10%). Your category evidence should exist as a list entry, a comparison row, and a definition. Engines that hedge toward criteria will still surface you if you're the criterion.
- Measure with repeats, never with screenshots. Given a self-agreement floor of 0.62, a single before-and-after comparison cannot distinguish a real win from Tuesday. Run each prompt at least three times per engine, report presence rate rather than position, and hold prompt wording, locale and session state constant between measurements — changing any of them invalidates the comparison.
- Re-baseline quarterly. Engines ship retrieval and model updates independently. Pair-level agreement in our tracking moves within weeks, so a Q1 source list is a stale target by Q3.
Common questions
Is it a bug that AI engines give different answers to the same prompt?
No. Different answers are the expected output of different retrieval systems feeding different models. Two engines can share an index and still diverge — Google AI Mode and AI Overviews agreed on only 39% of brand names in our test despite both being Google. Treat divergence as a measurement problem, not a defect.
Why does the same engine give me a different answer than it gave yesterday?
Because engines are not deterministic and their source pool changes daily. Same-engine self-agreement across three runs averaged 0.62 in our data — roughly four in ten brand slots changed between identical runs of the same prompt. Sampling randomness, index refresh, and silent model updates all contribute. A day-over-day change is not evidence that anything you did worked.
Which AI engine should my brand optimize for first?
Start with whichever engine your buyers actually use, then check your retrieval coverage there. If you have to pick blind, the Google systems and ChatGPT carry the most B2B volume, but they have the slowest source refresh — median citation age of 16 and 9 months. Work started there takes longest to show up, which is an argument for starting sooner, not for skipping it.
How many times should I run a prompt before trusting the result?
At minimum three, ideally five, per engine. Same-engine self-agreement averaged 0.62 across three runs in our data. A single run tells you what happened once; it does not tell you your ai share of voice.
Does getting cited more automatically get me recommended more?
Not automatically. In 23% of divergence events, the engine cited a page that mentioned the brand and still didn't recommend it. Citations get you into consideration; a clear, self-contained description of what you do and who you serve is what gets you into the recommended set.
Can I make engines agree with each other?
Not fully, and it isn't the goal. Divergence is structural — different indexes, different crawl cadence, different endorsement caps. What you can move is your floor: brands on consensus pages appeared in a median of 5 of 8 engines versus 2 for single-engine brands. Aim to raise the number of engines that name you, not to make their lists identical.
Do these numbers change over time?
Yes, and faster than traditional search rankings. Engines ship retrieval and model updates independently, so pair-level agreement moves in weeks. The four-layer structure has held steady across our tracking, but the specific percentages should be re-measured quarterly against your own category rather than assumed.
The takeaway
Eight engines give eight different answers because they are eight different two-stage systems, and both stages vary. The useful insight isn't that they disagree — it's that 54% of the disagreement is about what they read and 46% is about how they reason, and those need separate budgets, separate tactics, and separate reporting.
Teams that treat AI visibility as one number will keep chasing noise. Teams that decompose it by layer will know, on any given week, whether they have a source problem or a story problem.