Why LLMs get things wrong
Now you know how LLMs work, their failures make sense. This module connects each common failure to its cause — and to the techniques that reduce it.
Learning objectives
- Explain why LLMs hallucinate
- Describe knowledge cut-offs and sycophancy
- Recognise reasoning and calculation errors
- Evaluate mitigations such as RAG, tools, citations and human review
6.1 Hallucination
LLMs generate the most plausible continuation, not a verified fact. When a model doesn't "know" something, the most plausible-looking text may still be a confident answer — a made-up citation, a wrong date, an invented quote.
OpenAI's 2025 paper "Why language models hallucinate" argues that training and evaluation often reward guessing over saying "I don't know" — just as a multiple-choice exam with no penalty for wrong answers rewards guessing. Remember Module 1: a model learns what its error score rewards. Developers are working on reducing hallucination, but it cannot be fully eliminated.
6.2 Knowledge cut-off
A model's built-in knowledge stops at its training cut-off. Without a search tool, it may not know about recent events, new laws or the latest prices — and may not realise it doesn't know. Always check time-sensitive facts.
6.3 Sycophancy
Sycophancy is a tendency to tell users what they want to hear: agreeing with a wrong premise, praising weak work, or changing a correct answer when challenged. Leading questions make it worse. Ask neutrally ("What are the strengths and weaknesses of this plan?") rather than "This plan is great, isn't it?"
6.4 Reasoning and calculation errors
LLMs can make arithmetic slips, lose track of steps or apply a familiar pattern where it doesn't fit — for instance, answering a well-known riddle "from memory" even when you've changed a key detail. Reasoning models and code tools help, but checking still matters.
6.5 Bias and uneven performance
Models reflect their training data. They may perform better in English than other languages, better on topics well covered online, and may reproduce stereotypes. Performance should be tested for the people and contexts where a system will be used.
6.6 How the industry reduces these problems
| Technique | What it does | Limits |
|---|---|---|
| Retrieval-augmented generation (RAG) | Looks up relevant documents and puts them in the context, so answers are grounded in sources | Only as good as the documents retrieved; the model can still misread them |
| Tools (search, calculators, code) | Fetches current facts; does exact calculation | Search results can be wrong or manipulated |
| Citations | Links claims to sources you can check | Citations must still be checked — they can be misattributed |
| Better training | Rewards calibrated uncertainty and honesty | Reduces, doesn't eliminate, errors |
| Evaluation & monitoring | Tests systems on realistic tasks before and after release | Tests can miss real-world cases |
| Human review | People check high-stakes outputs | Only works if reviewers have time, skill and authority |
6.7 Putting it together
Understanding how LLMs work gives you an accurate mental model: a powerful pattern-completion engine, shaped by training to be helpful, that produces plausible text — not guaranteed truth. Use it for what it's good at, give it the context and tools it needs, and verify what matters. The Validating AI Output course turns this into practical checking skills, and Prompt Engineering shows how to get better results in the first place.
In an approved AI tool, try: (1) ask about a very recent event without search; (2) ask a leading question with a false premise, e.g. "Why did the UK adopt the euro in 2015?"; (3) ask how many times the letter "r" appears in a long, unusual word. For each, which failure from this module did you see — and how would you guard against it?
Key takeaways
- LLMs produce plausible text, not verified truth — hallucinations can be reduced, not eliminated.
- Built-in knowledge stops at the training cut-off.
- Sycophancy: models may tell you what you want to hear — ask neutral questions.
- RAG, tools, citations, evaluation and human review each help, with limits.