From rules to learning
Every AI system you use rests on one shift in thinking: instead of telling computers the rules, we let them learn the rules from examples.
Learning objectives
- Contrast traditional programming with machine learning
- Define data, features, labels, models, training and inference
- Describe training intuitively as error-reducing adjustment
- Explain why AI advanced so quickly from the 2010s
1.1 Two ways to make a computer do something
For most of computing history, software worked one way: a programmer wrote explicit rules. That's perfect when the rules are known — calculating VAT, sorting a list, booking a seat.
But some tasks resist rules. Try writing instructions for recognising a cat in a photo: pointy ears (but some cats' ears fold), whiskers (but so do dogs), fur (but so do rugs)… You'd never finish. Machine learning flips the approach: instead of writing rules, you show the computer many examples and let it find the patterns itself.
Traditional programming
Rules + data → answers. A person writes the rules; the computer applies them.
Machine learning
Data + answers → rules. The computer learns the "rules" (a model) from examples; then it applies the model to new data.
1.2 The building blocks
| Term | Plain-English meaning | House-price example |
|---|---|---|
| Data | The examples the system learns from | 10,000 past house sales |
| Features | The pieces of information about each example | Bedrooms, floor area, postcode, garden |
| Label | The answer you want it to learn to predict | The sale price |
| Model | The learned pattern — a mathematical function with adjustable settings (parameters) | "Each extra bedroom adds roughly £X in this area…" |
| Training | Adjusting the parameters so predictions match the examples as closely as possible | Tuning until predicted prices are close to real ones |
| Inference | Using the trained model on new data | Estimating the price of a house that's just been listed |
1.3 How training works, intuitively
Imagine tuning a radio. You turn the dial, listen, and turn it a bit more in whichever direction sounds clearer. Training is similar, but with thousands to billions of dials (parameters):
- The model makes a prediction for a training example.
- We measure how wrong it was, using a loss (error) score.
- We nudge every parameter slightly in the direction that would have reduced the error.
- Repeat — over every example, many times.
This nudging process is called gradient descent. You don't need the maths to use AI well, but the intuition matters: a model only learns what its data and its error score reward. That idea will come back when we look at why LLMs hallucinate.
1.4 Why now?
The core ideas behind machine learning and neural networks are decades old. Three things came together in the 2010s to make them powerful:
- Data: the internet and digitisation created huge datasets of text, images and behaviour.
- Compute: graphics processing units (GPUs), originally built for video games, turned out to be excellent at the maths neural networks need.
- Algorithms: better techniques for training very large networks — culminating in the transformer architecture in 2017.
For each task, decide whether rules or machine learning fits better: (a) calculating holiday pay, (b) detecting fraudulent insurance claims, (c) checking a form has a valid date, (d) transcribing a phone call.
Show answers
(a) Rules — the policy is known. (b) Learning — fraud patterns are complex and change. (c) Rules. (d) Learning — speech varies far too much for hand-written rules.
Key takeaways
- Machine learning learns rules from data and answers instead of being given rules.
- A model is a function with parameters adjusted in training to reduce error.
- A model only learns what its data and error score reward.
- Data, compute (GPUs) and better algorithms drove the AI boom.