AI Visibility Audit a reproducible brand visibility audit

AI visibility audit resources

Practical, source-backed guides for the four core tasks of any reproducible audit: building a prompt sample, preserving answer evidence, checking citations, and comparing repeat runs. Use these alongside the full method and the worked example.

Build a prompt sample

A prompt sample is a fixed, bounded set of questions you will ask an AI assistant. It is the foundation of a reproducible audit because it defines exactly what you are testing. Build it before you run any queries, and do not change it mid-audit.

Steps

  1. Define your categories. At minimum, include brand questions (e.g., “What is [brand]?”), category questions (e.g., “Which tools help with [problem]?”), and problem questions (e.g., “How do I solve [problem]?”).
  2. Write concrete, natural prompts. Use the phrasing a real user might type, not an artificially precise query. Avoid leading language that steers the answer.
  3. Set inclusion and exclusion criteria. Decide in advance which answers count as “mentions” and which do not. For example, a mention might require the brand name to appear in the main answer text, not just in a footnote.
  4. Fix the number of prompts. A small, focused sample (e.g., 10–20 prompts) is easier to manage and review than a large one. You can always expand later.
  5. Record the sample. Save the exact prompt text, the order, and any parameters (temperature, model version, etc.) in your evidence ledger.

For guidance on how AI features appear in search results and how structured data is treated, see Google’s documentation on AI features in Search and structured data policies. These are primary sources for understanding the ecosystem, but they do not guarantee any specific outcome for your brand.

Preserve answer evidence

Every observation must be captured verbatim and traceable. A screenshot alone is not enough; you need the full text, the model identifier, the date, and the settings used. This is what makes your audit reproducible.

What to record for each answer

  • Prompt text — the exact string you submitted.
  • Model and version — the exact identifier exposed by the provider, plus the capture date; do not infer an unavailable version.
  • Date and time — including timezone.
  • Settings — temperature, top-p, system prompt, or any other parameters you set.
  • Full answer text — copy the complete response, including any citations, footnotes, or disclaimers.
  • Source of the answer — if the assistant cites a source, record the URL and the exact claim it supports.

Store the evidence in a format that is easy to diff later, such as plain text or Markdown. Avoid pasting into a word processor that may silently alter formatting.

Check citations

When an AI assistant cites a source, verify that the citation is real, relevant, and correctly attributed. This is a critical quality check because models can hallucinate or misattribute sources.

Verification steps

  1. Open every cited URL. Confirm it exists and is accessible.
  2. Check the claim. Does the cited page actually support the statement the assistant made? Look for the specific sentence or data point.
  3. Note the date. A citation may be outdated or superseded. Record the publication date of the source.
  4. Flag missing citations. If the assistant makes a factual claim without a source, note that as a gap.
  5. Distinguish primary from secondary. Prefer primary sources (official documentation, research papers) over secondary summaries.

For understanding how structured data can help search engines understand your content, see schema.org and Google’s structured data policies. These are reference materials, not a guarantee that your content will be cited or ranked.

Compare repeat runs

AI assistants are non-deterministic. Running the same prompt twice can produce different answers. Comparing repeat runs is how you measure consistency and identify volatility.

How to compare

  • Run each prompt at least three times in a single session, and ideally across different days.
  • Record each run separately in the ledger, not just the first or the “best” one.
  • Diff the answers. Look for changes in brand mention, stance, and citation presence.
  • Calculate simple metrics. For example, “brand mentioned in 4 of 6 runs” or “citation appeared in 2 of 3 runs”.
  • Report the range. State the minimum and maximum scores you observed, not just an average.

This approach gives you a realistic picture of how your brand appears, rather than a single snapshot that may be an outlier.

Controlled fixtures vs. actual observations

It is essential to separate illustrative examples from real measurements. A controlled fixture is a sample you create to demonstrate a format or a scoring rule. An actual observation is a real answer you captured and recorded. Never present a fixture as if it were a measurement.

Label clearly

In any report, mark each example as either illustrative fixture or actual observation. Include the date and model for actual observations. This prevents confusion and keeps the audit honest.

Evidence ledger template

Use this preformatted template to record each observation. Copy it into a plain-text file and fill in the fields.

# AI Visibility Audit — Evidence Ledger
# One entry per prompt run.

## Run ID: [e.g., 2025-06-01-001]
- Date (UTC): [YYYY-MM-DD HH:MM]
- Model: [e.g., GPT-4o, Claude 3.5 Sonnet]
- Settings: [temperature, top-p, system prompt, etc.]
- Prompt text:
  [exact prompt]
- Answer text (verbatim):
  [full answer, including citations]
- Citations found:
  - [URL 1] — [claim supported]
  - [URL 2] — [claim supported]
- Brand mention: [yes/no]
- Stance: [positive/neutral/negative/not mentioned]
- Notes: [any anomalies, e.g., refusal, error, or unusual phrasing]

Review checklist

Before you publish or act on an audit, run through this checklist.

  • Prompt sample is fixed and recorded.
  • Every answer is captured verbatim with date, model, and settings.
  • All citations were verified against the actual source.
  • Repeat runs are included and reported as a range.
  • Illustrative fixtures are clearly labeled and separated from actual observations.
  • No claims of guaranteed ranking or citation are made.
  • The evidence ledger is complete and reproducible by another person.

Reasonable limits

Every audit has limits. Acknowledge them openly so your results are not overinterpreted.

Sampling and model volatility

Your prompt sample is a small subset of all possible questions. Answers vary by model, version, and even by run. A single audit cannot represent the entire landscape.

Personalization

Many AI assistants personalize answers based on user context, location, or history. Your observations may differ from what another user sees. State the context you used.

Causal uncertainty

An audit measures what an assistant says at a point in time. It does not prove why the assistant said it. Do not infer causation from correlation, and do not claim that any action you took caused a change unless you have a controlled experiment.

These limits are part of the method, not a flaw. A transparent audit is more useful than a confident black box.

Next steps

Read the full method for the complete scoring rubric, or follow the worked example to see the process in action. Both are designed to be reused and adapted to your own category.