Why AI Makes Things Up, and What Helps Against It

Key Takeaways

• A language model writes the most plausible continuation of a text. It doesn't check whether that continuation is true.
• Hallucinations happen mostly where the model lacks knowledge: rare facts, internal details and anything after its training data ends.
• You can't switch them off completely, but sources, clear tasks and permission to say “I don't know” make them much rarer.
• Always check numbers, dates, names and references against the original.
• Plan for hallucinations the way you plan for human error: with a process where someone checks before anything goes out.

AI makes things up because a language model produces the most likely continuation of a text rather than looking up checked facts. When it lacks a piece of information, it often fills the gap with something that sounds right: an invented source, a wrong date, a contract clause that doesn't exist. You can't prevent this entirely. You can make it rarer and easier to spot by giving the AI your sources, allowing it to say “I don't know”, asking for evidence and having a person check anything important.

What is an AI hallucination?

A hallucination is a statement that sounds convincing and is wrong. The model delivers it in the same confident tone as a correct one, which is exactly what makes it tricky. Typical examples from office life: a legal article with the wrong number, a court ruling that never existed, a price that isn't on any list, or meeting minutes recording a decision nobody made. The word is a metaphor. The model isn't trying to deceive you. It produces text that fits statistically. And hallucinations aren't limited to facts: a model can also misread a document you gave it, attribute a statement to the wrong person or quietly drop a condition. The summary then sounds right and still says something different from the original.

A language model sounds just as sure when it's guessing as when it knows.

Why does AI make things up?

Because the model is trained to produce plausible text, not checked knowledge. Several causes are behind this, and they often work together:

• It predicts text. A language model learns from huge amounts of text which word tends to follow which. Anything that follows regular patterns, like grammar and spelling, it handles very well. Rare, arbitrary facts such as a particular person's birthday or your internal notice period can't be worked out from patterns, though.
• Guessing gets rewarded. A research paper published by OpenAI in 2025 argues that training and common benchmarks reward models for guessing rather than for admitting uncertainty. As in a multiple-choice exam, a guess scores more on average than a blank.
• Knowledge is missing. The model doesn't know your business and knows nothing about what happened after its training data ends. If you don't give it the information, guessing is all it has.
• The prompt pushes it. Ask for “five sources” and you'll get five, even if only two exist. Vague or leading prompts raise the risk.
• Long texts. With very long documents, the model can miss details or mix up numbers from different places.

When are hallucinations most risky?

When wrong information travels on unnoticed and does damage further down the line. That applies above all to:

• Numbers, amounts, deadlines and dates.
• Legal articles, rulings, standards and other references.
• Names, addresses, account numbers and contract details.
• Anything that goes to customers, authorities or partners without a check.
• Questions that go beyond the documents you provided.

Your internal knowledge deserves particular care. A general AI assistant knows nothing about your clients, prices or agreements, yet it will still answer questions about them readily. Questions like these belong in a tool that searches your own documents, or with a person who knows.

What helps against hallucinations?

1. Give it sources.Have the AI answer from your documents rather than from memory. On a small scale, that means uploading the document to the chat. On a larger scale it's called RAG, which we explain in RAG explained.
2. Allow “I don't know”.Say explicitly in your instructions that the AI should tell you when the documents don't answer the question. That alone makes invented answers much less common.
3. Ask for evidence.For every important statement, ask for a word-for-word quote or a page reference. Whatever can't be backed up comes out. That makes the answer checkable.
4. Keep the task narrow.Summarising, pulling out details or sorting things into categories works more reliably than open knowledge questions. The clearer the task and the format, the less room there is to invent.
5. Leave sums and lookups to the right tools.Prices come from your price list, totals from the spreadsheet, deadlines from your system. Where a fixed rule will do, it's more reliable than any AI, as we explain in AI or rule-based automation.
6. Have a person check.For anything that leaves the building or supports a decision, someone with the expertise reads it through. It takes minutes and prevents the expensive mistakes.

Some tools offer a setting called temperature. A low value makes answers more consistent and predictable. It doesn't make up for missing knowledge, though: a model that doesn't know the answer will simply guess more consistently.

How do you write a prompt that leaves less room to guess?

Name the source, ask for evidence and say what should happen when something is missing. Here's an example for a question about a contract: “Answer the question using only the attached contract. For every statement, give the clause number you're relying on and quote the sentence word for word. If the contract doesn't cover the question, write: not covered in the contract. Don't add anything from general knowledge.” An instruction like this takes away the pressure to deliver an answer at any cost, and it makes every statement checkable in seconds.

Tools that search the web or your files and link to their sources don't take the checking off your hands entirely either. A link only proves that the source exists. Whether it says what the answer claims, you only find out by opening it. There's more on writing good instructions in prompting basics.

How can your team check AI answers in daily work?

With one simple rule everyone knows: anything containing a number, a date, a name or a source gets checked against the original. An example: a project lead at a construction firm has an AI summarise a forty-page tender. That saves the first read-through. The deadlines, penalty clauses and quantities still get looked up in the document before anyone prices the job. Rules like this belong in your AI policy and in your team's training. Our checklist AI guardrails without the red tape is a good starting point, and our team training page shows how we teach teams to work this way.

Will better models make hallucinations disappear?

Not on their own, and it's not worth waiting for. The research paper mentioned above finds that hallucinations persist even in the most advanced systems, and providers themselves describe measures like the ones above as ways to reduce hallucinations, not to rule them out. So plan for them the way you plan for human error: with a process that checks at the right points. Then AI is useful even for important work. If you'd like to work out where that helps your business most, get in touch.