When a human must decide
Validation tells you whether output is good. Judgement tells you whether a person, not a system, should make the call.
Learning objectives
- Identify decisions that require human judgement
- Describe legal expectations of human involvement in the UK and EU
- Distinguish meaningful oversight from rubber-stamping
- Apply a proportionate validation checklist
- Document checks and build a checking culture
6.1 Some decisions need a person
Even perfectly validated AI output shouldn't make some decisions on its own. Human judgement is essential when a decision:
- significantly affects someone's life — jobs, money, health, housing, education, legal status;
- involves values, ethics or trade-offs between people's interests;
- needs accountability — someone who can explain and answer for it;
- is novel or ambiguous, outside what the AI was designed or tested for;
- is hard to reverse.
6.2 The legal backdrop
Law increasingly expects meaningful human involvement:
- UK and EU data protection law gives people protections around significant decisions based solely on automated processing, including rights to human intervention and to contest decisions. The UK's Data (Use and Access) Act 2025 updated these rules, but safeguards remain for significant decisions.
- The EU AI Act requires high-risk AI systems to be designed for effective human oversight, and deployers to assign oversight to people with the necessary competence, training and authority.
6.3 Meaningful, not rubber-stamp, oversight
Rubber stamp
A reviewer approves 200 AI decisions an hour, can't see the inputs, has no time to investigate and is never expected to disagree.
Meaningful oversight
A trained reviewer sees the inputs and the AI's reasoning, has time and authority to override, is expected to challenge, and their overrides are tracked to improve the system.
A useful signal: if the human never disagrees with the AI, the oversight may not be real.
6.4 A validation checklist
Bring the course together with this checklist. Scale it to the stakes.
- Purpose: Is this a task AI should be doing? Is a human decision needed? (Modules 1 and 6)
- Facts: Are claims, names, dates and figures verified at independent, authoritative sources? (Module 3)
- Sources: Do citations exist and say what's claimed? (Module 3)
- Context: Does it match the documents provided? Is it current? (Module 2)
- Logic and numbers: Does the reasoning hold? Have key numbers been recomputed? (Module 5)
- Fairness: Could this treat anyone unfairly? Did you run a counterfactual check where relevant? (Module 4)
- Completeness: What's missing — caveats, risks, other views?
- Accountability: Are you comfortable putting your name to it? Is AI use disclosed where it should be?
6.5 Document what you checked
For important work, keep a brief record: what AI was used for, what you checked and how, and who signed it off. This protects you, helps colleagues trust the work, and supports your organisation's governance.
6.6 Build a team culture of checking
- Make it normal to ask "how was this checked?"
- Share examples of caught errors — without blame
- Set clear review levels for different types of AI-assisted work
- Give reviewers time; checking is real work
Take an AI-generated piece of work from your job (or ask for one: e.g. "a one-page briefing on the UK's approach to AI regulation, with sources"). Apply the full checklist. Record each error you found, its type (Module 2), and how you found it.
Key takeaways
- Significant, value-laden, novel or irreversible decisions need people.
- Data protection law and the EU AI Act expect meaningful human involvement in key cases.
- If reviewers never disagree, oversight may not be real.
- Use the checklist: purpose, facts, sources, context, logic, fairness, completeness, accountability.
- Record what you checked; make checking normal.