Types of machine learning
Not all machine learning works the same way. Knowing how a model learned — and how it was tested — tells you how much to trust it.
Learning objectives
- Distinguish supervised, unsupervised, reinforcement and self-supervised learning
- Explain why models are tested on unseen data
- Recognise overfitting and underfitting
- Explain why accuracy alone can mislead, using precision and recall
2.1 Three ways machines learn
| Type | How it learns | Typical uses |
|---|---|---|
| Supervised learning | From labelled examples — inputs paired with the right answers. | Spam detection, price prediction, medical image classification |
| Unsupervised learning | From unlabelled data, finding structure on its own. | Customer segmentation, anomaly detection, topic discovery |
| Reinforcement learning | By trial and error, receiving rewards or penalties for actions. | Game-playing, robotics, and fine-tuning LLMs with human feedback |
Supervised learning splits further into classification (predicting a category — spam or not) and regression (predicting a number — a price). Large language models start with self-supervised learning: the "label" is simply the next word in real text, so no one has to label anything by hand. That's what lets them learn from such enormous amounts of data.
2.2 Testing a model honestly
A model that scores perfectly on the examples it trained on may still be useless. What matters is how it performs on data it has never seen. So data is split:
- Training set — used to learn the parameters.
- Validation set — used to compare versions and tune settings.
- Test set — kept locked away until the end, for an honest final score.
2.3 Overfitting and underfitting
Overfitting
The model memorises the training data, including its noise and quirks — like a student who memorises past exam answers but can't handle a new question. Great training score, poor real-world performance.
Underfitting
The model is too simple to capture the real pattern — like predicting every house costs the national average. Poor everywhere.
The goal is generalisation: performing well on new, real-world data.
2.4 Measuring performance
"Accuracy" — the share of predictions that are right — can mislead. If 1 in 1,000 transactions is fraudulent, a model that says "not fraud" every time is 99.9% accurate and completely useless. Practitioners use other measures too:
- Precision: of the cases the model flagged, how many were truly positive? (Low precision = lots of false alarms.)
- Recall: of all the truly positive cases, how many did the model catch? (Low recall = missed cases.)
There's usually a trade-off. For cancer screening you might accept more false alarms to miss fewer cases; for a spam filter, you'd rather let some spam through than lose an important email.
2.5 Garbage in, garbage out
A model can only be as good as its data. If the training data is incomplete, outdated, wrongly labelled or unrepresentative of the people it will be used on, the model inherits those flaws — often invisibly. Performance should be checked not just overall but for different groups of people and situations.
For each system, decide whether you'd prioritise precision (fewer false alarms) or recall (fewer misses): (a) an airport security scanner, (b) an email filter that auto-deletes "spam", (c) a tool that flags possible child-safeguarding concerns for human review.
Show suggested answers
(a) Recall — missing a threat is far worse than a manual bag check. (b) Precision — auto-deleting a real email is costly. (c) Recall — humans review the flags, so catching more cases matters more than some false alarms.
Key takeaways
- Supervised learning uses labelled examples; unsupervised finds structure; reinforcement learns from rewards.
- LLMs are pre-trained with self-supervised learning on next-token prediction.
- Honest evaluation uses held-out test data.
- Overfitting means memorising rather than generalising.
- Precision and recall reveal what accuracy hides.