Transparency, explainability and accountability
People can't trust, use well or challenge AI they can't see or understand. This module covers the practices that make AI open and answerable.
Learning objectives
- Describe levels of AI transparency
- Use documentation tools such as model cards and system cards
- Compare interpretable models and post-hoc explanations
- Apply the ICO's six types of explanation
- Put accountability mechanisms in place
4.1 Transparency: being open about AI
Transparency means being open that AI is used, what it does, its limitations and how it's governed. It operates at different levels:
- Disclosure: telling people when they're interacting with AI or seeing AI-generated content (required in some cases by Article 50 of the EU AI Act).
- System information: what the system is for, how well it performs and where it doesn't work well.
- Organisational openness: publishing policies, impact assessments or transparency reports.
4.2 Documentation tools
| Tool | What it documents |
|---|---|
| Model cards | A model's intended use, performance (including across groups), limitations and ethical considerations. Proposed by Google researchers in 2019; now widely used. |
| Datasheets for datasets | How a dataset was collected, its composition, intended uses and known issues. |
| System cards | Published by frontier AI developers such as Anthropic and OpenAI: capabilities, safety evaluations and mitigations for a model release. |
| Transparency notes | Microsoft's documents explaining how its AI services work, intended uses and limitations, to help customers deploy them responsibly. |
4.3 Explainability: why did it decide that?
Explainability is the ability to give meaningful reasons for an AI output. It matters most when decisions affect people, so they can understand, trust or challenge them.
- Interpretable models (e.g. simple decision trees, scorecards) are explainable by design. For high-stakes decisions, consider whether a simpler model would do.
- Post-hoc explanation methods (e.g. feature-importance techniques) estimate which inputs drove a complex model's output. They're useful but approximate.
- LLM self-explanations aren't guaranteed to reflect the model's actual internal process, as interpretability research has shown.
4.4 Explaining decisions to people
The UK ICO and The Alan Turing Institute's guidance Explaining decisions made with AI describes six types of explanation organisations may need to provide:
| Explanation type | Answers |
|---|---|
| Rationale | Why did the system reach this decision? |
| Responsibility | Who is involved and who can I talk to for a human review? |
| Data | What data was used, and how? |
| Fairness | What steps were taken to make it fair? |
| Safety and performance | How accurate, reliable and secure is it? |
| Impact | What effect does this have on me and on society? |
4.5 Accountability
Accountability means identifiable people and organisations answer for an AI system's design, use and effects — and that affected people have routes to redress. In practice:
- A named owner for every AI system in use
- Clear decision rights: who can approve, change or switch off a system
- Records: documentation, risk assessments and logs
- Contestability: a way for people to challenge decisions and get human review
- Audit: internal or external checks that the system behaves as claimed
AI systems often involve a model developer, a platform, a system integrator and a deploying organisation. When something goes wrong, each can point to the others. Contracts, documentation and laws such as the EU AI Act — which assigns duties to providers and deployers — help pin responsibility down.
Key takeaways
- Transparency covers disclosure, system information and organisational openness.
- Model cards, datasheets, system cards and transparency notes document AI.
- Consider interpretable models for high-stakes decisions; post-hoc explanations are approximate.
- ICO explanation types: rationale, responsibility, data, fairness, safety and performance, impact.
- Every AI system needs an owner, decision rights, records, contestability and audit.