LLM tracking: how to monitor what AI says about your brand

LLM tracking means checking, on a schedule, what AI assistants such as ChatGPT, Gemini, Perplexity and Google's AI Overviews say when your buyers ask about your category. Done well, it shows whether you are recommended, how you are described and which sources shape the answer. Done badly, it reports noise as a ranking.

By AI Search Ranking. Published 12 November 2026. Last updated 12 November 2026. In AI visibility measurement.

On this page

The short answer

LLM tracking is the routine of asking AI assistants the questions your buyers ask, recording the answers, and watching how they change. A large language model (LLM) is the system behind each assistant. The aim is to know three things: whether you are named, whether you are described accurately, and which sources the answer relies on.

The method matters more than the software. Write a fixed set of real buyer questions. Check each assistant separately, from the country you sell in. Run every question more than once, because the same question can produce a different answer minutes later. Then set those results beside the free data from Google and Bing, and your own analytics.

This guide sets out that routine step by step. It also covers the two mistakes that make most tracking misleading: trusting a single run, and reporting a "position" in an answer as if it were a Google ranking.

What LLM tracking covers, and what it does not

The term is used for two different jobs. Engineers use "LLM monitoring" to mean watching their own AI application for errors, cost and quality. Marketers use "LLM tracking" to mean watching what public assistants say about a brand. Google's own AI Overview for this search ends by asking whether you mean brand visibility or tracking the models themselves. This guide is about the second.

For brand tracking, four things are worth recording, and no single source shows all of them.

What you want to know	What to record	Where it comes from
Are we recommended?	Whether the answer names you, and how often across runs	Your own prompt checks
Are we described correctly?	What the answer says about you, word for word	Your own prompt checks
Which sources shape the answer?	The pages and domains the answer cites	Your own prompt checks, Bing's report
Are real people seeing us?	Impressions, citations and visits	Google Search Console, Bing Webmaster Tools, analytics

Our guide to LLM visibility tools compares the products that automate the first three rows. Everything below works with or without one.

Write the prompt set

Your prompt set is the list of questions you will track. It decides everything that follows, so write it from how buyers really ask, not from your keyword list.

Category questions. "What is the best payroll software for a UK small business?" These show whether you make the shortlist at all.
Use case questions. "Which CRM suits a firm that sells to other businesses on a long sales cycle?" These test whether you are matched to the buyers you want.
Comparison questions. "How does [your brand] compare with [competitor]?" These show how you are positioned against named rivals.
Problem questions. "How do I stop losing leads between marketing and sales?" These test whether the assistant reaches for you before the buyer knows your category.
Brand questions. "Is [your brand] any good?" and "What does [your brand] cost?" These catch errors in how you are described.

Take the wording from sales calls, enquiry forms, support tickets and the People also ask questions for your main searches. Keep a few close rewordings of your most important questions. People phrase the same need in very different ways, and the assistant's answer can shift with the wording.

Once the set is written, freeze it. Add new questions as a separate group, so the core set stays comparable from month to month.

Decide where and how to ask

Each assistant draws on different sources and behaves differently, so check each one separately. A blended score hides where you are strong and where you are missing.

Set the conditions and keep them the same every time:

The assistants. Start with the ones your buyers use. For most UK businesses that means ChatGPT, Google's AI Overviews and AI Mode, Gemini and Perplexity, with Copilot and Claude where relevant.
The country and language. UK answers differ from US ones. Ask from the UK, in English, or set the location in whatever service runs the checks.
A clean session. Use a signed-out or fresh account with no memory or chat history. A personalised answer tells you about one user, not the market.
The date and time. Record when each check ran. Assistants change their models and sources without notice, and the date is what lets you link a change to its cause.

Run every question more than once

This is the step most tracking skips, and it changes the result more than any other.

AI assistants do not return a fixed answer. In a study published by SparkToro, 600 volunteers ran 12 identical prompts through ChatGPT, Claude and Google's AI a combined 2,961 times1. The chance of the same list of brands appearing in any two responses was under 1 in 1001. The chance of the same list in the same order was nearer 1 in 1,0001.

The same study found that presence held up better than order. Some brands appeared in most responses for a given question, even though their place in the list moved about1. Its conclusion: the share of answers that name you, across many questions and runs, is a reasonable measure1. A "ranking position" in an AI answer is not1.

So run each core question several times per check, and report how often you were named, not where you appeared. Treat the first item in a list as a detail, never as the headline.

Our test: how stable were ChatGPT's answers to UK buyer questions?

On 8 November 2026 we asked ChatGPT four everyday UK buyer questions. They covered accounting software, a CRM for a small company selling to other businesses, payroll software, and a business bank account. Each question ran three times, from the UK, in separate sessions. We also asked one reworded version of each, once.

Four things stood out.

The core shortlist was steadier than the large studies suggest. In these narrow UK categories, a small group of brands appeared in every run. The accounting answer named the same five products, in the same order, every time, including when we reworded it. That fits the SparkToro finding that smaller markets produce more stable answers1.

The edges moved. Outside that core, brands came and went. In the payroll answers, budget and free options appeared in some runs and not others. In the CRM answers, two larger platforms appeared in only some runs. If your brand sits at the edge of a category, one run can show you as present or absent by chance.

The first pick changed on one question. The CRM question named one product first in two runs and a rival first in the other. Rewording the payroll question changed which product came first. A tracker that reports "position one" from a single run would have reported a change that was really noise.

The sources moved more than the brands. For three of the four questions, fewer than half the cited domains appeared in all three runs. Track the sources separately from the brands, and over several runs, before deciding which publications shape your category.

Our test was small: one assistant, one day, four questions. It is a reason to run your own checks more than once, not a benchmark. The raw answers, sources and counts are saved with this article's research data.

Log the same fields every time

A tracking record is only useful if every check is logged the same way. A simple spreadsheet works. For each run, record:

Field	Example
Date and time	8 November 2026, plus the time of the run
Assistant and mode	ChatGPT, signed out, web search on
Country and language	United Kingdom, English
Prompt	The exact wording, from the frozen set
Named?	Yes or no
Recommended or only mentioned?	"Best for small teams" or "also available"
What it said about you	The sentence, copied word for word
Errors	Wrong price, old product name, wrong location
Cited sources	Every URL in the answer
Competitors named	In the order given

Keep the full answer text as well, or a dated screenshot. When a figure moves, you need to be able to open the answers behind it.

Add the data that comes from real users

Prompt checks measure test questions. Three free sources measure real exposure, and they keep the prompt checks honest.

Google Search Console. Google launched generative AI performance reports on 3 June 20262. The Search report counts impressions of your pages in AI Overviews and AI Mode, and groups them by page, country, device and date3. Google's help page says the report reached all websites worldwide by 31 August 20263. It does not list a query view, so it shows which pages appear but not which questions triggered them.

Bing Webmaster Tools. Its AI Performance report shows how often your content is cited in Microsoft Copilot, Bing's AI summaries and some partner services4. It also lists sample grounding queries: the phrases the AI used when it retrieved your pages4.

Your analytics. Visits from assistants often arrive as referrals. In our test, about half the source links in ChatGPT's answers carried a tag naming chatgpt.com as the source. Links straight to brand websites usually carried none, so treat analytics as a floor, not a full count.

None of these covers every assistant. Together they tell you whether the changes in your prompt checks are reaching real people.

Set a cadence and report trends

Answers vary from run to run, so a single check is a sample, not a result. A practical rhythm for most businesses:

Monthly: run the full prompt set, several times per question, in each assistant. Compare with the previous month and with your baseline.
Weekly, if you have a reason: recheck a short list of high-value questions after a launch, a price change or a correction you are waiting to see.
Quarterly: review the prompt set itself. Add questions buyers have started asking, but keep the core set unchanged.

Report three figures per assistant: the share of runs that name you, the share that describe you accurately, and the sources that appear most often. Show competitors on the same basis. Leave out average position.

Act on what you find

Tracking is only worth the effort if it changes the work.

An error in how you are described. Find the likely source: your own pages, an old directory listing, a review site, a dated article. Fix it at the source, then recheck the question over the following weeks.
A question where you never appear. Check whether you have a page that answers it directly and clearly. If not, that is a content gap.
A source that keeps appearing. A publication or comparison site that is cited again and again is where your digital PR and listings work should start.
A Google surface where you are missing. Google says there are no extra technical requirements for AI Overviews or AI Mode5. A page must be indexed and eligible to show a snippet in Search5. So a missing page is often an ordinary indexing or crawling problem first.

That last point corrects a claim common in tracking guides: that technical SEO hardly matters for AI visibility. For Google's AI features, crawling and indexing are the entry ticket5.

This is the loop our generative engine optimisation (GEO) service runs. It uses a fixed prompt set, checks each assistant separately and runs each question more than once. Results are reported beside the client's own numbers, then we fix the sources behind the answers. Our guide to generative engine optimisation covers the fixes in more depth, and our pages on AI Overviews and ChatGPT cover each platform.

Common questions

What is LLM tracking? It is the practice of checking, on a schedule, what AI assistants say when buyers ask about your category. You record whether you are named, how you are described and which sources are cited, then follow the trend over time.

Can I track my brand in ChatGPT for free? Yes, by hand. Ask your core questions in a signed-out session, several times each, and log the answers in a spreadsheet. Add Google Search Console and Bing Webmaster Tools, which report real AI exposure on their own surfaces at no cost3 4.

How often should I check AI answers? Monthly suits most businesses, with each question run more than once per check. Check weekly only when you are waiting to see the effect of a specific change.

Is tracking my position in AI answers useful? Not on its own. Research found that the order of brands in AI answers changes almost every time a question is asked1. How often you are named, across many runs, is the more reliable measure.

Is LLM monitoring the same as LLM tracking? Not quite. LLM monitoring often means watching your own AI application for errors, cost and quality. LLM tracking, in the marketing sense, means watching what public assistants say about your brand.

Where to start

Write twenty real buyer questions this week, drawn from your sales calls and enquiries. Run each one three times in ChatGPT and in Google, signed out, from the UK, and log the answers. Then switch on the free AI reports in Search Console and Bing Webmaster Tools. That baseline tells you where you stand and what to fix first.

If you would rather have the tracking and the fixes handled together, talk to us about our GEO service. It starts with that prompt set, checked in each assistant, and a plan to become a source they cite.

What we've seen in real audits

We have tried many of the AI tracking tools now on the market, and in our experience none of them yet gives fully reliable data on what assistants say. We still use them for what they do well. When a business has hundreds of prompts to watch, a dedicated platform shows the broad pattern and helps us understand the audience. For the questions that matter most, though, we go into ChatGPT, Perplexity and Google ourselves and run the same prompts on a fixed routine. We set those results beside the AI referral traffic in Google Analytics, so we can see whether being mentioned actually brings visitors. We have also pulled AI answers through a data provider's API, which makes regular checks easier to repeat. The tools give the overview; the manual checks show what a buyer actually sees.