Brand presence flip rate by platform. Share of consecutive answer pairs where the tracked brand appeared in one answer and not the other.

Bodhium Labs · 9 minute read · Data from 20 July to 14 September 2026

A brand appears in an AI answer on Monday. On Tuesday the same question gets an answer without it. The brand's website did not change. No competitor published anything. The question was the same.

We have argued before that one AI answer is weak evidence, and that a content experiment needs repeated readings before a good result counts. Here we back that claim with what we observed. The data comes from the Bodhium Labs visibility platform, which puts the same questions to the AI platforms every day for the brands it tracks.

2.53M

AI answers observed

2.31M

pairs of consecutive answers to one question on one platform

24,410

questions tracked on the best-covered platform, across 250 brands

To protect our customers, we report only totals by platform. No brand, client, question, or brand-owned domain is named. The method notes at the end define each measure and list what we left out.

What we compared

We send each tracked question to several AI platforms on a schedule, usually once a day. So for each question and platform we hold a long series of answers. We took every pair of consecutive answers in that series and asked three things.

One pair of answers

Was the tracked brand present?In both, in neither, or in one

Which brands were named?Overlap of the two sets of brands

Which domains were cited?Overlap of the two sets of domains

Within a pair, the question, the platform, and the brand's setup stay fixed. Only the day changes. Overlap is Jaccard similarity: the size of the intersection divided by the size of the union. It runs from 0, nothing shared, to 1, identical.

The window runs from 20 July to 14 September 2026. We kept only days on which a question received exactly one answer on a platform. We also kept only brands whose entity graph stayed the same across the comparison. The entity graph is our record of which names and spellings count as a mention of a brand or one of its rivals. Had it changed midway, a change in what counts as “the brand” could pass for a change in the answer.

Brand mentions are fleeting

Start with the simplest question. In what share of pairs was the brand in one answer and not the other?

Figure 01

Brand presence flipped between consecutive answers

Share of consecutive pairs, by platform · 2,307,379 pairs

A flip means the brand was in one answer and not the other. Each bar covers every pair on that platform, including questions where the brand never appears or always does.

Between 5.8% and 9.6% of pairs flip, depending on the platform. That understates the effect. Most pairs come from questions where the brand always appears or never does. Conditioning on the previous answer tells us more. When the brand was present, how often had it gone by the next answer? When it was absent, how often had it arrived?

Platform	Brand present	Lost by next answer	Gained by next answer	Pairs
ChatGPT	30%	15%	7%	551,605
Gemini	35%	13%	7%	553,667
Perplexity	32%	10%	5%	587,674
Google AI Overview	34%	14%	7%	527,566
Claude	26%	11%	4%	86,867

“Lost” is the share of answers with the brand present whose successor lacked it. “Gained” is the share of answers without the brand whose successor included it.

Finding

A brand named in today's AI answer has a 10% to 15% chance of missing from tomorrow's answer to the same question. A brand absent today has only a 4% to 7% chance of turning up.

The question did not change between the two answers, and, as a later section shows, how long you wait makes little difference.

Most questions are settled. The rest are noisy.

Averages hide the shape of this. We took each question with at least twenty answers on a platform and sorted it into one of three groups: the brand never appeared, always appeared, or sometimes appeared.

Figure 02

How questions split by the brand's fate

Share of questions with 20+ answers, sampled one in four · by platform

One question in four was sampled to keep the per-question series manageable. Sample sizes are in the method notes.

Between 25% and 51% of questions sit in the middle group, and that group decides visibility. Within it, the flip rate rises to 19% to 24% per pair.

Finding

For the questions a brand can win or lose, about one AI answer in four or five disagrees with the one before it.

Branded questions, which name the brand, are far more settled than the rest. The table separates them.

Platform	Branded: present	Branded: flip	Non-branded: present	Non-branded: flip
ChatGPT	60%	7.9%	18%	9.7%
Gemini	62%	4.3%	24%	11.1%
Perplexity	62%	3.8%	21%	7.6%
Google AI Overview	63%	6.7%	23%	10.7%
Claude	97%	3.0%	14%	6.3%

The list of brands changes more than the brand's presence

A brand can be present or absent. The list of brands around it can change in more ways than that. For each pair we compared the full set of brands named in the two answers.

Figure 03

Overlap of the brands named in consecutive answers

Mean Jaccard similarity · pairs where at least one answer named a brand

1.0 means both answers named the same brands. Pairs where neither answer named any brand are left out.

On ChatGPT the mean overlap is 0.46.

Finding

Only 17% of consecutive ChatGPT answers to the same question name the same set of brands. Another 19% share no brand at all.

An answer that names your brand and three rivals today names a partly different list tomorrow. Any ranking built from one answer inherits that churn.

The sources churn faster still

In The Citation Revolving Door we followed pages after their first citation. Here is the same churn seen from one question, one day to the next.

Figure 04

Overlap of the domains cited in consecutive answers

Mean Jaccard similarity · pairs where at least one answer cited a source

Domains, not URLs, so two pages on the same site count as a match. The measure is generous to the platforms.

The averages hide a split. On ChatGPT, 50% of pairs share no cited domain at all. Perplexity cites far more sources per answer, 14 domains on average against 5 for ChatGPT, and its pairs fall into two camps: many share little, and many share most of their sources.

Figure 05

How cited-domain overlap is distributed

Share of pairs in each overlap band, 0 to 1 · by platform

ChatGPT

Gemini

Perplexity

Google AI Overview

Claude

Each panel is one platform. The leftmost bar is the share of pairs with no domain in common; the rightmost is the share whose two answers cite exactly the same domains.

Finding

Anyone who reads a citation as proof that a page “ranks” in AI search should weigh this. On every platform we measured, tomorrow's answer to the same question rests on a substantially different set of sources. On ChatGPT, half the time, it shares no source with today's.

Waiting longer does not change the answer more

The obvious explanation is that the web moves. Indexes refresh, pages change, answers follow. If that drove the churn, pairs a week apart should disagree more than pairs a day apart. They do not.

Figure 06

Flip rate by days between the two answers

Share of pairs where brand presence flipped, by gap in days · gaps with at least 500 pairs

Most pairs are one day apart, because enterprise brands run daily. The seven-day points come from weekly schedules and from gaps in daily series.

On ChatGPT the flip rate is 9.3% for answers a day apart and 8.4% for answers a week apart. The other platforms are just as flat. Most of the variation comes from how the platform composes an answer, not from changes in the world it describes. Two runs on the same day would, we expect, disagree about as often as two runs a week apart.

Finding

AI answers a week apart disagree no more often than answers a day apart. The churn belongs to the platform. Every single reading carries it, and waiting does not wash it out.

How many runs before the number means something

If one answer is noisy, the useful question is how many it takes. For each question in the middle group, we took its long-run mention rate as the truth and measured how far off a running estimate was after one answer, five, ten, and so on.

Figure 07

Error of a visibility estimate versus runs averaged

Mean absolute error, in percentage points, against the question's long-run mention rate · questions with 40+ answers

ChatGPT in green; other platforms in grey. Only questions where the brand sometimes appears are included. The others can be read from one answer.

Platform	Questions	After 1 run	After 5	After 10	After 20	Runs to ≤10 pts
ChatGPT	631	23	12	9	6	8
Gemini	626	23	12	10	7	9
Perplexity	577	31	18	14	10	22
Google AI Overview	796	31	20	13	8	15
Claude	82	22	14	11	7	12

Mean absolute error in percentage points. “Runs to ≤10 pts” is the first run count at which the mean error falls to ten points or below.

Finding

One AI answer misstates a question's visibility by 22 to 31 points on average. A dashboard that reports a change after one run is reporting the noise.

After ten answers the error is 9 to 14 points and still falling. A visibility number is therefore better reported as a range than as a single figure, with the width of the range set by how many answers stand behind it.

What this means for testing a page edit

This measurement sits under our simulator work. A live answer disagrees with itself about one day in ten on whether the brand appears, and on ChatGPT it shares no sources with the day before half the time. Publishing an edit and watching the live answers is therefore a slow experiment with little power. The simulator removes the retrieval churn. It answers a question from a fixed set of pages, so two runs differ only in how the answer is written, and an edit can be compared against a baseline over the same sources.

That helps only if the simulator behaves like the platform it stands in for, so we check it continuously. For a tracked set of questions, once the real ChatGPT answer arrives, the simulator answers the same question from the pages ChatGPT cited. A fixed judge then scores the two answers on the brands named, their order, and the claims made. We call the score fidelity and report it from 0 to 100.

Figure 08

Production simulator fidelity against real ChatGPT answers

2,802 scored comparisons · 99 tracked questions · 26 July to 15 September 2026

Each bar is the share of comparisons whose score fell in that band of ten. The orange line marks the median. Fidelity blends three parts: ranking agreement, agreement on the set of brands, and whether the claims in one answer follow from the other. The median is 77; the middle half of scores lies between 65 and 88. Comparisons with no fetchable cited page were not scored; there were 645 of them.

Week starting	Comparisons	Mean fidelity	Median
2026-07-27	196	73.8	76.4
2026-08-03	204	73.4	74.3
2026-08-10	201	74.9	76.3
2026-08-17	724	74.6	77.7
2026-08-24	461	75.6	78.1
2026-08-31	474	73.3	75.5
2026-09-07	431	75.7	78.3
2026-09-14	109	77.1	81.1

Two things stand out. The score holds steady week to week, which is what an instrument should do. And it is not 100. The weakest part is agreement on the set of brands, with a mean of 59. That is the same dimension on which the live platform disagrees most with itself. A fresh ChatGPT answer names a different list of brands from the previous one, and the simulator is scored against one such answer. So a fidelity score carries some of this noise itself. On another day the reference answer would have named a different list, and the same simulated answer would have scored differently.

The rule that follows is simple. Test a page edit in the simulator first, where the sources hold still and repeats are cheap. Then track the live answers for long enough for the estimate to settle, and treat the first week of readings as a small sample of a noisy process.

Method notes

Source. Our production daily tables, read on 15 September 2026. Answers from 20 July to 14 September 2026 inclusive. Coverage by platform:

Platform	Answers	Questions	Pairs	Answers citing ≥1 source	Brands named per answer	Domains cited per citing answer
ChatGPT	606,786	21,780	551,605	50%	4.3	5.5
Gemini	609,164	21,783	553,667	76%	4.7	9.8
Perplexity	645,760	24,410	587,674	74%	3.1	13.7
Google AI Overview	582,150	24,320	527,566	62%	3.4	7.2
Claude	89,004	2,105	86,867	70%	2.1	5.0

Pair. Two consecutive days on which the same question received exactly one answer on the same platform. Days with more than one answer, about 1.9% of daily metric rows in a sample week, were dropped, not merged.
Brand present. The brand's own entity, from its active entity graph, was matched in the answer. Brands without an active entity graph were excluded, so a change to the graph cannot show up as a change in the answer.
Brands named. The set of graph entities matched in the answer, within the active graph generation. Overlap is Jaccard similarity. Pairs where neither answer named a brand are left out of the mean.
Domains cited. The set of normalized cited domains for that question, day, and platform. Pairs where neither answer cited anything are left out of the mean.
Settled versus noisy. Computed on a fixed one-in-four sample of questions with at least 20 answers on the platform. Sample sizes: ChatGPT 3,160; Gemini 3,162; Perplexity 3,302; Google AI Overview 3,200; Claude 488.
Convergence. For sampled questions with at least 40 answers whose brand sometimes appeared, we compared the running mean after k answers with the mean over the whole series, and report the mean absolute difference across questions. Because the reference is the same series, the curve must reach zero at the series length. The shape over the first 30 runs is the point.
Gap. Days between the two answers in a pair. Bands with fewer than 500 pairs are not shown.
Simulator. Internal shadow runs on ChatGPT for a tracked question set, engine gpt-5-mini, judge model fixed, sources frozen at capture time. Scores are the stored fidelity values. Failed comparisons are excluded and their failure reasons are given above.
Not measured. Whether an edit changes a live answer, and how much of the churn comes from each platform's retrieval rather than its writing. Consumer web captures of ChatGPT and Gemini were measured but are not reported here.

Ask the Same Question Tomorrow: How Much AI Answers Change Between Runs

Karthik Shetty