Fine-tuning glossary
Key terms from this course, in plain English.
- Adapter
- A small set of trainable parameters added to a frozen model, as in LoRA.
- Base model
- The pre-trained model you start from before fine-tuning.
- Catastrophic forgetting
- Loss of general abilities when a model is trained narrowly.
- Checkpoint
- A saved copy of model weights at a point in training.
- Continued pre-training
- Further pre-training on unlabelled domain text before task tuning.
- Data leakage
- When test data influences training, inflating evaluation results.
- Distillation
- Training a small student model on outputs from a larger teacher model.
- DPO
- Direct Preference Optimization: tuning on preferred vs rejected responses.
- Epoch
- One full pass through the training data.
- Fine-tuning
- Further training a pre-trained model on your examples to adapt its behaviour.
- Full fine-tuning
- Updating all of a model's parameters.
- JSONL
- JSON Lines: a text format with one JSON object per line.
- Learning rate
- How large each training update step is.
- LoRA
- Low-Rank Adaptation: a PEFT method that trains small low-rank matrices alongside frozen weights.
- Model card
- Documentation of a model's purpose, data, performance, limitations and ownership.
- Model routing
- Sending each request to the most cost-effective model able to handle it.
- PEFT
- Parameter-efficient fine-tuning: methods that train only a small number of parameters.
- Preference tuning
- Training on human or AI preferences between responses.
- Prompt caching
- Reusing the processed form of a repeated prompt prefix to reduce cost and latency.
- Pruning
- Removing weights or components that contribute little to a model.
- QLoRA
- LoRA applied to a base model loaded in 4-bit precision.
- Quantisation
- Storing model weights with fewer bits to save memory and speed up inference.
- Reinforcement fine-tuning (RFT)
- Tuning a model using rewards from a grader on its answers.
- Supervised fine-tuning (SFT)
- Training on examples of inputs paired with ideal outputs.
- Validation loss
- Error on held-out validation data, used to detect overfitting.