When to fine-tune
Fine-tuning is the most over-prescribed technique in generative AI. This module helps you decide when it's genuinely the right tool.
Learning objectives
- Define fine-tuning
- Apply the optimisation ladder
- Identify what fine-tuning is good and bad at
- Compare hosted and self-managed fine-tuning
- Estimate the full cost of a fine-tuned model
1.1 What fine-tuning is
Fine-tuning means taking a pre-trained model and training it further on your own examples so it adapts its behaviour. The model's parameters change; afterwards, it behaves differently even with a short prompt.
1.2 Try these first
Fine-tuning is powerful but slow, costly to maintain and easy to get wrong. Most teams should climb this ladder one rung at a time, stopping as soon as quality is good enough:
- Better prompts — clear instructions, examples, structure (see Prompt Engineering).
- A better or different model — a stronger model, or a reasoning model, may simply solve it.
- Retrieval (RAG) — if the problem is missing knowledge (see RAG).
- Fine-tuning — if the problem is behaviour, format, style or efficiency, and you have good examples.
OpenAI's model optimisation guidance makes the same point: prompt engineering "may be all you need", and fine-tuning "can be a time-consuming process".
1.3 What fine-tuning is good — and bad — at
Good at
- Consistent output formats and structure
- Tone, style and brand voice
- Narrow classification and extraction tasks
- Following complex instructions reliably
- Making a smaller, cheaper, faster model do one task well
- Shorter prompts (fewer tokens per call)
Bad at
- Teaching frequently changing facts — use RAG
- Fixing a vague task definition — fix the task first
- Small datasets of mixed quality
- Guaranteeing accuracy — fine-tuned models still hallucinate
- Anything you can't evaluate
1.4 Where you can fine-tune
| Option | Examples | Trade-offs |
|---|---|---|
| Hosted fine-tuning from model providers and clouds | Cloud AI platforms such as Amazon Bedrock, Google Vertex AI and Microsoft Azure AI Foundry offer fine-tuning for selected models | Easy and managed; limited choice of models and methods; check data terms |
| Open-weight models you tune yourself | Llama, Mistral, Gemma, Qwen and others, using open-source libraries such as Hugging Face Transformers and PEFT | Full control and privacy; you manage GPUs, serving and security |
Provider offerings change often. In 2026, OpenAI's documentation states that its fine-tuning platform is winding down and no longer open to new users — a reminder to check current availability, and to design so you're not locked into one vendor's tuning service.
1.5 The total cost of a fine-tuned model
- Data: collecting, cleaning and labelling examples — usually the biggest cost.
- Training: compute (GPUs) or platform fees.
- Evaluation: test sets and repeated comparisons.
- Serving: hosting a custom model may cost more than calling a shared one.
- Maintenance: when the base model is updated or retired, you may need to re-tune and re-evaluate.
(a) Your support bot doesn't know about this month's new product line. (b) Your product descriptions must always follow a precise 5-part template and brand voice, and prompting gets it right 80% of the time. (c) You want to classify 2 million emails a day cheaply.
Show suggested answers
(a) RAG — it's missing knowledge. (b) Good fine-tuning candidate — consistent format and style, after strengthening the prompt. (c) Strong candidate — fine-tune (or distil into) a small, cheap model for one narrow task.
Key takeaways
- Fine-tuning changes the model's weights using your examples.
- Try prompting, a better model and RAG first.
- Fine-tuning suits format, style, narrow tasks and efficiency — not changing facts.
- Provider offerings change; avoid lock-in.
- Data and maintenance are usually the biggest costs.