AI Hallucination: What It Is and How to Manage It

A hallucination is a model stating something false with the same confidence it uses for something true. The word makes it sound like a malfunction. It is not. It is the system working exactly as designed, on a question where the design does not help.

A language model generates plausible text. Plausible and true are different properties that happen to overlap most of the time, because the text it learned from was mostly true. Where they come apart, nothing in the model notices.

Where it shows up

Citations and case law are the worst, because the shape of a citation is highly predictable and the content is not. A model can generate a perfectly formatted reference to a paper that does not exist. Numbers are next: figures, dates, dosages, statute numbers, prices. Then anything about a specific named entity the training text barely covered, which includes almost every small business, including yours.

What actually reduces it

Give it the source. Retrieval puts your documents in front of the model before it answers, so it is summarising something real rather than reconstructing from memory. This is the single largest reduction available and the reason serious deployments are built around retrieval rather than around the model alone.

Ask for the reasoning. A model that shows its steps produces errors you can see. A model that returns a conclusion produces errors that look like conclusions.

Ask it to argue the other side. Models agree too readily. Pushing back on a correct answer will often make one fold, so asking whether you are right is close to useless. Asking it to make the opposing case is not.

Constrain the answer space. Extraction from a supplied document fails far less often than open-ended generation, because there is less room to invent.

What does not work

Telling it not to hallucinate. Telling it to only state facts. Telling it to say "I don't know" when unsure. None of these do anything, because the model has no reliable internal measure of its own certainty to act on.

The rule worth adopting

Verify anything you would be embarrassed to get wrong, and build the verification into the workflow rather than relying on someone remembering. A control that depends on vigilance is not a control.