Glossary

AI hallucination

Written with AI from the sources below and checked against our facts sheet. Last checked 7 Nov 2026. How this site is made · Spot an error?

Part of: NIST AI Risk Management Framework

An AI hallucination is fluent, confident output from a generative AI model that is false or made up, such as an invented court case. NIST calls it confabulation. It happens because models predict plausible text rather than check facts, so governance manages it with grounding, testing, human review and honest claims.

In plain English

Picture a student who never says "I don't know." Asked something they can't answer, they produce a smooth, plausible reply with invented details. A generative model behaves the same way, because it predicts plausible output rather than verified truth.

The confident tone is what makes it dangerous. People believe fluent answers. Models can even produce made-up citations or reasoning steps that seem to justify a wrong answer, as NIST AI 600-1 notes.

Hallucination is a generative AI failure. A fraud classifier that misjudges a transaction is wrong, but it is not hallucinating.

Example

Calloway Health, a fictional hospital group, tests an assistant that drafts discharge summaries. In testing, one summary lists a penicillin allergy that is not in the patient's record.

The governance team responds in layers:

Grounding. The assistant may draw only on the patient's record, and each statement must cite its source line.
Human review. A nurse checks every summary before it enters the chart.
Testing. A set of 500 past records measures the rate of unsupported statements, against a threshold agreed before testing.
Monitoring. Clinicians can flag errors with one click, and the rate is reviewed monthly.
Honest claims. Calloway's materials describe the assistant's limits and never call it "hallucination-free."

Why it matters

NIST. The Generative AI Profile (July 2024) lists confabulation among 12 risks unique to or worsened by generative AI. It maps the risk to the AI RMF characteristics, including valid and reliable, safe, and explainable and interpretable. The AI RMF asks for accuracy measured on test sets that reflect real conditions of use.

EU AI Act. The Act never says "hallucination." High-risk systems must reach an appropriate level of accuracy, declared in their instructions for use (Art. 15). General-purpose AI model providers must tell downstream providers about capabilities and limitations (Art. 53(1)(b)).

Consumer protection. Accuracy claims need evidence. In 2024 the Texas Attorney General settled with Pieces Technologies over allegedly inaccurate claims about its clinical AI's hallucination rate. Overstating reliability can become AI washing.

How the AIGP exam tests it

Hallucination appears as a trait of probabilistic AI (lesson I.A.3) and as an incident cause (lesson III.C.8).

Common traps:

A label is not a root cause. "Model hallucination; added a line to the system prompt" closes nothing. If errors began after a content update, suspect stale or conflicting documents.
Wrong failure type. A claims classifier that goes wrong as the world changes shows drift, not hallucination.
Impossible guarantees. No vendor can promise 100% accuracy. Governance sets error tolerances, thresholds and human review.
Grounding as a cure. Retrieval-augmented generation reduces hallucination but does not remove it.

The tell: if a generative system invents something, think confabulation; then ask why it happened.

Frequently asked questions

Why do AI models hallucinate?

Generative models produce outputs that approximate the statistical patterns in their training data. A language model predicts likely next words; it does not look facts up. That usually yields accurate text, but it can also yield fluent falsehoods. NIST calls confabulation a natural result of how generative models are designed. Open-ended questions in specialist domains raise the risk.

Can AI hallucinations be eliminated?

Not with current generative models. They can be reduced. Common controls include grounding answers in trusted documents, requiring citations and testing on realistic questions. Low-confidence or high-stakes outputs go to a person, and error reports are monitored. Governance sets an acceptable error rate for the use case instead of demanding perfection.

What is the difference between hallucination and confabulation?

They mean the same thing. NIST's Generative AI Profile uses confabulation for confidently stated but erroneous or false content, and notes that hallucination is the colloquial term. NIST adds that some commenters see hallucination as anthropomorphizing AI. Expect both words in AIGP materials.

Does retrieval-augmented generation stop hallucinations?

It reduces them but does not stop them. Retrieval-augmented generation gives the model relevant documents at query time, so answers can be grounded and cited. The model can still misread or ignore a source. If the documents are stale, wrong or poisoned, it will produce confident wrong answers that look well sourced.

Sources

Spot an error? Tell us