Module 4 of 6 · 35 minutes

Running a fine-tune

Time to train. This module walks through a real LoRA fine-tune and the few settings and signals that decide whether it works.

Learning objectives

Run a supervised LoRA fine-tune with open-source libraries
Explain learning rate, epochs, LoRA rank and batch size
Diagnose overfitting from training and validation loss
Describe the hosted fine-tuning workflow
Recognise forgetting and safety erosion risks

4.1 A LoRA fine-tune with open-source tools

This example uses Hugging Face's open-source libraries — transformers, peft (for LoRA) and trl (for supervised fine-tuning) — on a small open-weight model so it can run on a single cloud GPU (e.g. a Colab GPU runtime). Library APIs evolve; check each library's documentation for current arguments.

Python · supervised fine-tuning with LoRA# !pip install transformers peft trl datasets accelerate
from datasets import load_dataset
from peft import LoraConfig
from trl import SFTTrainer, SFTConfig

BASE = "an-open-instruct-model"         # e.g. a small instruction-tuned model whose licence allows tuning

data = load_dataset("json", data_files={"train": "train.jsonl", "validation": "val.jsonl"})

lora = LoraConfig(
    r=16,                 # rank of the low-rank update: capacity vs size
    lora_alpha=32,        # scaling of the update
    lora_dropout=0.05,
    target_modules="all-linear",
    task_type="CAUSAL_LM",
)

args = SFTConfig(
    output_dir="gfy-descriptions-lora",
    num_train_epochs=3,
    learning_rate=2e-4,
    per_device_train_batch_size=4,
    eval_strategy="epoch",          # report validation loss each epoch
    logging_steps=10,
)

trainer = SFTTrainer(model=BASE, args=args, peft_config=lora,
                     train_dataset=data["train"], eval_dataset=data["validation"])
trainer.train()
trainer.save_model("gfy-descriptions-lora")   # saves the small adapter, not the whole model

The fine-tuning workflow, end to end.

4.2 The settings that matter most

Setting	What it controls	If too high / too low
Learning rate	How big each update step is	Too high: unstable, forgets; too low: barely learns
Epochs	How many passes over the data	Too many: overfits and memorises; too few: underfits
LoRA rank (r)	Capacity of the adapter	Too high: more memory, overfitting risk; too low: can't capture the task
Batch size	Examples per update	Limited by GPU memory; affects stability

Change one setting at a time, and keep a log of every run and its results.

4.3 Reading the loss curves

Watch training loss and validation loss together. Both falling is good. If training loss keeps falling while validation loss starts rising, the model is overfitting — memorising the training set. Stop at (or roll back to) the checkpoint with the lowest validation loss.

Training loss keeps falling, but validation loss turns upward when the model starts memorising. Keep the checkpoint where validation loss was lowest.

4.4 Hosted fine-tuning

With a managed service, the steps are the same but simpler: upload training and validation files in the required format, choose a base model and a few hyperparameters, start the job, and monitor the reported losses. You then call the tuned model by its new identifier. The data preparation and evaluation work doesn't get any smaller.

4.5 Catastrophic forgetting and safety erosion

Narrow tuning can degrade a model's general abilities — and research has shown that fine-tuning can weaken safety behaviours learned in post-training, even with benign data. Always re-check general capability and safety after tuning (Module 5), and prefer PEFT methods and modest training where possible.

Key takeaways

LoRA fine-tuning needs a dataset, a base model, a LoRA config and a trainer.
Learning rate and epochs are the most influential settings.
Rising validation loss with falling training loss means overfitting.
Hosted services simplify training but not data or evaluation.
Re-check general ability and safety after tuning.