Statistical Visibility Test

Know how many prompts it takes to trust your AI mention rate.

AI models answer the same question differently every time, so a quick spot-check tells you almost nothing. Set your targets below and get the exact number of prompts your visibility test needs before you spend budget on one that is too small.

Your exact prompt count

A defensible sample size for your confidence and precision targets.

Your real request budget

See the total API calls up front so you can scope cost before you run a single test.

No black boxes

Every statistical assumption is a slider you control, so you can defend the number to anyone.

Test parameters

• Confidence level: 80% / 90% / 95% / 99%
• Margin of error: ±0.5% / ±2% / ±4% / ±5%
• Resamples per prompt (K): 1 / 3 / 5 / 10
• Expected mention rate: 50%
  Roughly how often you expect your brand to appear. 50% is the safe, most-conservative default.
• Prompt-to-prompt variance: 0.020
  How much mention rates differ across your prompts. Higher variance needs more unique prompts. Leave near 0.02 if unsure.

Your sample size is ready

What the statistical visibility test does

Before you can improve your brand's presence in AI answers, you have to measure it accurately. The Statistical Visibility Test tells you how big that measurement has to be to mean something.

Language models are inconsistent. Ask the same question twice and you can get your brand in one answer and a competitor in the next. Run a tiny test of a dozen prompts at one attempt each, and your mention rate is closer to a coin flip than to data. Decisions built on that number usually go wrong.

So the planner treats this like any serious measurement. It accounts for two kinds of noise: the model's run-to-run variation on a single prompt, which you reduce by asking each prompt a few times, and the real spread between different prompts. From your precision target, it calculates the number of prompts that makes the result reliable. Every assumption stays visible as a slider you control, because a number you can question is a number you can trust.

How it works

1. Pick how precise

   Choose the margin you can act on. A ±2% window works for most brand-visibility decisions.

2. Pick how certain

   Set your confidence. 95% is the working standard. Reach for 99% when the decision is expensive to get wrong.

3. Set the repeats

   Choose how many times to repeat each prompt. Repeating smooths out the model's run-to-run variation.

4. Get your number

   Read the exact prompt count and total request budget, recalculated the instant you touch a control.

Frequently asked

What does 'AI mention rate' actually mean?

It is the share of relevant AI answers in which your brand shows up or gets recommended. Appear in 40 out of 100 answers to category questions and your mention rate is 40%. It is the closest thing AEO has to a rankings number.

Why can't I just ask ChatGPT a few times and count?

The model's answer changes from run to run, and your prompts differ from each other. A handful of tries can land far from the real rate, so you end up steering strategy by luck. Sizing the test first turns a hunch into a measurement.

Which confidence level is right for me?

95% covers most decisions. Drop to 90% for a fast directional read. Step up to 99% when the call carries real cost. Higher certainty always asks for more prompts.

How tight should my margin be?

For marketing decisions, ±2% to ±3% is usually enough. At a true rate of 40%, a ±2% margin means you will land somewhere between 38% and 42%. A tighter band multiplies the work quickly.

Why repeat the same prompt several times?

One prompt asked once inherits all of the model's randomness. Ask it three to five times and average the results, and that single prompt becomes a far steadier reading. It is the cheapest noise reduction available.

What's the math behind the number?

It is a standard proportion sample-size formula, extended for repeats. The estimate's variance is (spread between prompts + p(1-p) ÷ repeats), and we solve for the prompt count that pulls your margin down to target at the chosen confidence.

How often should I re-run the test?

Monthly testing keeps you ahead of model updates and shifting competitor content. Quarterly is the floor for real tracking. Monthly suits an active program.

Is it really free?

Yes. Set your targets, add a work email once, and your required sample size shows and updates live as you move the controls. No account, no cost.