Why responsible AI matters
Responsible AI grew out of real harm to real people. This module looks at what went wrong — and what organisations learned.
Learning objectives
- Distinguish AI ethics, responsible AI and AI governance
- Describe landmark cases of AI harm
- Identify common causes across those cases
- Explain why responsible AI matters to organisations
- Recognise new concerns raised by generative AI
1.1 What "responsible AI" means
Responsible AI is the practice of designing, building, buying and using AI in ways that are safe, fair, transparent, accountable and respectful of people's rights — throughout the system's life. It sits alongside two related ideas:
| Term | Focus |
|---|---|
| AI ethics | The moral questions: what should we build, for whom, and with what trade-offs? |
| Responsible AI | Turning ethical principles into practice — processes, tools, roles and habits. |
| AI governance | The organisational structures, policies and controls that make sure it happens, and meet legal duties. (See AI Governance, Risk & Compliance.) |
1.2 Lessons from real failures
Responsible AI isn't theoretical. These well-documented cases shaped the field:
In 2018, Reuters reported that Amazon had scrapped an experimental CV-screening tool after finding it penalised CVs containing the word "women's" (as in "women's chess club captain"). It had learned from ten years of CVs submitted mostly by men.
In 2016, ProPublica analysed COMPAS, a tool used in US courts to predict reoffending, and reported that Black defendants who did not reoffend were far more likely to be wrongly labelled high-risk than white defendants. The tool's maker disputed the analysis. The debate revealed that different, reasonable definitions of "fair" can conflict (Module 3).
In 2020, with exams cancelled due to COVID-19, an algorithm was used to moderate A-level grades in England. Many students — disproportionately at larger state schools — were downgraded from their teachers' predictions, while small cohorts, more common in independent schools, were less affected. After public outcry, the government reverted to teacher-assessed grades.
In the Dutch childcare benefits scandal, tens of thousands of families were wrongly accused of fraud, with risk-profiling that included nationality as a factor. The scandal led to the government's resignation in January 2021.
The 2018 "Gender Shades" study by Joy Buolamwini and Timnit Gebru found that commercial gender-classification systems were far less accurate for darker-skinned women than for lighter-skinned men. Several vendors subsequently improved their systems.
1.3 Common threads
- Historical data encoded past unfairness.
- Testing missed the groups most affected.
- People affected couldn't understand or challenge decisions.
- Accountability was unclear — "the algorithm decided".
- Harms fell hardest on people with least power to push back.
1.4 Why it matters for organisations
- People: real harm to individuals and communities.
- Law: equality, data protection, consumer protection and AI-specific laws such as the EU AI Act.
- Trust: customers, employees and the public increasingly ask how AI is used.
- Quality: responsible practices — testing, documentation, monitoring — usually produce better, more reliable products.
1.5 Generative AI raises new questions
Generative AI adds concerns that earlier "predictive" AI didn't: hallucinated content, deepfakes and misinformation, copyright and creators' rights, harmful content generation, over-reliance, and environmental costs of training and running large models. The principles in this course apply to all of them.
Key takeaways
- Ethics asks what we should do; responsible AI puts it into practice; governance makes sure it happens.
- Landmark failures share causes: biased data, poor testing, opacity, unclear accountability.
- Harm often falls on those with least power.
- Responsible practice protects people, meets the law, builds trust and improves quality.