Hallucination
huh-loo-sih-NAY-shun
When an AI model generates confident but factually incorrect output. It sounds right. It reads well. It is wrong.
A hallucination is when an AI model generates something that sounds correct but is factually wrong. The model does not know it is wrong. It has no concept of truth. It predicts the next most likely token based on patterns in its training data. Sometimes those patterns produce false information delivered with complete confidence.
This is the fundamental trust problem with LLMs. Ask Claude for a list of academic papers on a topic and it might invent papers that do not exist, complete with plausible-sounding titles and author names. Ask it for a company's pricing and it might give you numbers from two years ago, or numbers it made up entirely.
For marketers, hallucinations are both a risk and an opportunity. The risk: an LLM might tell a developer your product does something it does not do. The opportunity: well-structured, factual content reduces hallucinations about your product. If your documentation clearly states your pricing, features, and limitations, the model has accurate source material to draw from. RAG is the most effective technical solution for reducing hallucinations. AEO is the marketing strategy for ensuring your content is represented accurately.
Examples
An LLM invents a product feature.
A developer asks ChatGPT: "Does Supabase support GraphQL?" The model confidently responds with instructions for enabling GraphQL in Supabase. But Supabase does not natively support GraphQL. The model generated a plausible but false answer.
A legal AI hallucinates case citations.
A lawyer used ChatGPT to research case law and submitted a brief citing six cases. None of them existed. The model invented case names, docket numbers, and rulings. The lawyer was sanctioned by the court. This happened at Levidow, Levidow & Oberman in 2023.
Reducing hallucinations with RAG.
A company's AI assistant was hallucinating product specs 15% of the time. They added RAG, grounding every response in retrieved documentation pages. Hallucination rate dropped to 2%.
In practice
Read more on the blog
Frequently asked questions
Can hallucinations be completely eliminated?
No. Hallucinations are an inherent property of how LLMs work. They generate text based on statistical patterns, not factual understanding. You can reduce hallucinations dramatically with RAG, better prompts, and confidence thresholds, but you cannot eliminate them entirely.
Related terms
A neural network trained on massive text data to generate and understand language. The technology behind ChatGPT, Claude, and Gemini.
Fetching relevant data and feeding it to an LLM so the response is grounded in real, current information instead of training data alone.
Training an existing model on your specific data to improve its performance on your tasks. Customization without building from scratch.
Optimizing content to be discovered and cited by AI systems. The AI-era equivalent of SEO, but for LLMs instead of search engines.
Want the complete playbook?
Picks and Shovels is the definitive guide to developer marketing. Amazon #1 bestseller with practical strategies from 30 years of marketing to developers.