Hallucination

A hallucination is output from a large language model (LLM) that is fluent and confident but not supported by its input, its retrieved context or the facts. Examples include an invented refund policy, a made-up command flag or an unsupported root cause. The model gives no built-in signal that it is guessing.

How it works

An LLM generates likely next tokens, not verified facts, so gaps in its context get filled with plausible text. In RAG (retrieval-augmented generation), weak retrieval raises the risk. In our workshop, relevant questions had a median retrieval similarity of about 0.5 to 0.6, while out-of-scope questions fell to about 0.1 to 0.2. Faithfulness checks compare the answer with the retrieved context.

Why it matters in production

A hallucinated answer returns HTTP 200 with normal latency, so no operational metric moves. Catching it needs evaluation scores, user feedback and prompts that say to answer only from context. In incident tooling, a confident unsupported cause is worse than saying the evidence is missing.