Research · Accuracy benchmark
Gibraltar AI Accuracy Benchmark
Do AI assistants know the facts about Gibraltar businesses, or make them up? We asked each one directly and graded every answer against the facts.
2026-09-pilot · measured · 36 real and 9 invented businesses · 720 answers graded
56%
Asked about a Gibraltar business, AI assistants get every fact right in 56% of answers.
23% state something wrong, and asked about businesses that do not exist, they describe them as real 19% of the time.
56%
of answers fully correct
23%
state a wrong fact
18%
say they do not know
19%
describe an invented business as real
Which assistants know Gibraltar businesses best?
The share of answers about real businesses with every fact right, by assistant and by whether it searched the web.
- ChatGPT, web search82%
- Gemini, web search78%
- Google AI Mode75%
- Perplexity74%
- Claude, web search65%
- Gemini, own knowledge31%
- ChatGPT, own knowledge28%
- Claude, own knowledge17%
Do they make businesses up?
Asked about businesses that do not exist, how often each assistant described one as real, with details, instead of saying it did not know.
- Gemini, own knowledge39%
- Gemini, web search28%
- ChatGPT, web search22%
- ChatGPT, own knowledge22%
- Perplexity17%
- Google AI Mode11%
- Claude, own knowledge11%
- Claude, web search6%
Which facts do they get right?
Of the answers asked for a business's contact details, the share giving the right phone number, website and address, and the share giving a wrong phone number.
| Assistant | Phone right | Website right | Address right | Wrong phone |
|---|---|---|---|---|
| ChatGPT, web search | 83% | 92% | 83% | 11% |
| Gemini, web search | 89% | 81% | 89% | 11% |
| Google AI Mode | 89% | 81% | 83% | 8% |
| Perplexity | 75% | 89% | 81% | 8% |
| Claude, web search | 81% | 86% | 69% | 11% |
| Gemini, own knowledge | 14% | 36% | 22% | 64% |
| ChatGPT, own knowledge | 3% | 3% | 0% | 3% |
| Claude, own knowledge | 0% | 0% | 0% | 0% |
What does this mean for a Gibraltar business?
- An assistant can name you and still give a customer the wrong number or address. Check what each one says about you.
- Where the facts come from matters: an old directory listing or Google profile is what the assistants repeat.
- Make your details plain on your own website and consistent everywhere else, so there is one version for them to find.
How does the benchmark work?
We ask each assistant two questions about each business, the way a customer would: what it is, and what its address, phone number and website are. We ask ChatGPT, Claude and Gemini twice, from their own knowledge and after searching the web, and Perplexity and Google AI Mode once. The businesses are drawn from every sector we track, alongside invented ones.
Each expected answer is broken into small criteria and an answer is scored by the share it meets. Facts that can be checked exactly are checked by code; the rest are read by a judge model against the reference facts. The full method is on our methodology page.
Common questions
Where do the reference facts come from?
From each business's Google listing: its address, phone number, website and the kind of business it is. They are reference facts, not verified by the business, so where an assistant disagrees we record the disagreement rather than assume the assistant is wrong about everything.
What are the invented businesses?
Restaurants that do not exist, with names checked against every business we track and against the web. The only right answer to a question about one is that the assistant does not know it. Describing one as real is a hallucination, and we count it.
How are answers graded?
Each expected answer is broken into small criteria: the right kind of business, the right street, the right phone number, the right website, nothing that contradicts the facts. Phone numbers, websites and streets are checked by code; whether an answer recognises the business and describes it correctly is read by a judge model against the reference facts.
Is declining to answer counted as wrong?
No. An assistant that says it does not know a business is recorded as declined, separately from one that gives a wrong phone number. For a business owner the second is worse: a customer who is told the wrong number does not ring you.