SEARCH VISIBILITY

How to Keep an AI Search Test Consistent from Week to Week

Use a run sheet to hold prompts and test conditions steady, then label every change that breaks comparability.

By Cuebase · Published · Updated

AI answers vary even when you change nothing. If you also change the question, country, model, or interface, you cannot tell which change moved the result. Your run sheet is the record that separates a repeat from a new experiment.

Fix the conditions before you run

FieldKeep fixedWhy it matters
PromptThe exact shopper wordingAdding a budget, size, or brand name asks a different question.
Market and languagePrompt wording plus provider location/language settings“UK” in text is not proof that the provider used a UK location.
Surface and providerConsumer website, API, or a provider-mediated APIThese are different products and may return different answers.
Engine and modelThe named engine and returned or configured modelA model change starts a new comparison series.
Capture and scoringPlanned repeats, timing pattern, tracked brands, and scoring rulesKeep the sampling plan fixed. Log actual dates, completions and failures for each run.

What Cuebase records

In Share of Voice, add your fixed queries and competitors, then choose Run probe now. Cuebase runs active queries through the engines available on your plan and records brand mentions, tracked-domain citations, and first appearance among your selected brands. Export CSV produces metric rows for saved probes: run time, query, category, engine, brand, mention, citation, and position.

The normal report does not give you a raw-answer viewer or a source-by-source answer export. If your study needs the full answer or source annotations, collect them in a separate, permissioned research record and label that method. Do not represent a manual evidence log as a standard Cuebase export.

A completed illustrative run

This is invented data. It shows how to complete the sheet, not a Cuebase result or a recommendation about any store.

FieldIllustrative valueComparable next week?
Prompt“linen sheets for hot sleepers in the United Kingdom”Yes, if unchanged.
SurfaceDataForSEO AI Optimization LLM Responses APIYes, if the same surface remains in use.
Engine / modelPerplexity / sonarYes, unless the configured or returned model changes.
Market and geolocationUK appears in prompt text; provider geolocation not configured.Yes, if both remain the same. This is not a UK-geolocated test.
Capture14 October 2026, 3 planned results, 2 completed, 1 provider failureCompare rates using 2 completed results and report the failure.

Changes that break the series

Mark the next run as a new series when you change the prompt, add or remove a tracked brand, change the tracked domain, switch the engine or model, change provider location or language, move from an API to a consumer interface, or alter the scoring rule. You can still keep both records. Just do not draw a week-over-week conclusion across the boundary.

A failure is different. If the setup is unchanged and one provider call fails, keep the failure in the sheet and calculate the rate from completed results. Do not count a failure as a zero mention. Rerunning only the failed request creates a different request pattern; record that retry.

Use the run sheet

Download the blank run sheet (CSV). Download the completed illustrative example (CSV). Before each run, copy the prior row, confirm every fixed field, and write a short comparability decision: “same series,” “new series,” or “not comparable.”

For metric definitions, read mentions, citations, and product links and your first AI visibility baseline. Use the AI share-of-voice guide for the product’s reporting context. See paid AI visibility tracking →