Module 6 of 6 · 30 minutes

Interpreting and deploying models

A model is only valuable if people can trust it, use it and keep it working. This module covers what happens after the accuracy score.

Learning objectives

Measure feature importance with permutation importance
Analyse errors by group
Save and load models safely
Plan deployment, monitoring and retraining
Choose between classic ML and LLMs

6.1 Which features matter?

Permutation importance measures how much a model's score drops when one feature's values are randomly shuffled — breaking its link with the target. It works for any model and is computed on held-out data.

from sklearn.inspection import permutation_importance

rf = RandomForestClassifier(n_estimators=300, random_state=0).fit(X_train, y_train)
imp = permutation_importance(rf, X_test, y_test, scoring="roc_auc", n_repeats=10, random_state=0)
print(pd.Series(imp.importances_mean, index=features).sort_values(ascending=False).round(3))

On our synthetic data you should see tenure_months and support_tickets near the top — matching the hidden pattern. On real data, importance tells you what the model relies on, which is not necessarily what causes the outcome.

Permutation importance: the model leans on tenure and support tickets — matching the hidden pattern in the practice data. (Illustrative values.)

6.2 Error analysis by group

test = X_test.copy()
test["actual"] = y_test
test["pred"] = (rf.predict_proba(X_test)[:, 1] >= 0.5).astype(int)
test["region"] = customers.loc[X_test.index, "region"]

for region, g in test.groupby("region"):
    print(region, "n =", len(g),
          " recall =", round(recall_score(g["actual"], g["pred"]), 2),
          " precision =", round(precision_score(g["actual"], g["pred"], zero_division=0), 2))

Check performance for every group that matters — regions, customer types and, where lawful and appropriate, protected characteristics. Overall scores can hide poor performance for smaller groups (see Responsible AI, Module 3).

6.3 Saving and loading a model

import joblib

joblib.dump(rf, "churn_model_v1.joblib")
loaded = joblib.load("churn_model_v1.joblib")
print(loaded.predict_proba(X_test.head(3))[:, 1].round(2))

Only load model files from sources you trust: like Python pickles, joblib files can execute code when loaded.

6.4 Deploying and monitoring

Serve predictions in batch (e.g. a nightly churn list) or in real time (an API)
Monitor input data for drift, prediction distributions, and real outcomes as they arrive
Retrain on a schedule or when performance drops — and re-evaluate before replacing the live model
Version data, code and models so any prediction can be traced
Document the model in a model card: purpose, data, metrics by group, limitations, owner

A model in production is never finished: monitor it, retrain it, and re-evaluate before each new version goes live.

6.5 Classic ML or an LLM?

Classic ML is usually better when…

you have structured, tabular data
you need a number or category at high volume and low cost
you have labelled historical outcomes
you need well-understood, auditable behaviour

An LLM is usually better when…

inputs are unstructured text, images or audio
you have few or no labelled examples
the task needs language generation or reasoning
requirements change often and prompting is faster than retraining

Many strong systems combine both — for example, an LLM extracts structured fields from emails, and a gradient-boosted model scores them.

Exercise: a mini model card

Write a short model card (as a Python dictionary or Markdown) for the churn model: purpose, training data, test AUC, recall by region, known limitations, and owner.

Show a solution

card = {
    "model": "churn_model_v1 (random forest, 300 trees)",
    "purpose": "Rank customers for weekly retention calls; human decides who to call",
    "data": "Synthetic customers, 2,000 rows; features: tenure, spend, tickets, promo",
    "test_auc": round(roc_auc_score(y_test, rf.predict_proba(X_test)[:, 1]), 3),
    "limitations": "Synthetic data; not validated on real customers; association not causation",
    "owner": "Retention analytics team",
}
print(card)

Combining both: an LLM turns unstructured text into features; a classic model scores them cheaply and consistently.

Key takeaways

Permutation importance shows what a model relies on — not causation.
Check metrics for every group that matters.
Only load model files from trusted sources.
Monitor drift and outcomes; retrain and re-evaluate.
Classic ML suits tabular, high-volume prediction; LLMs suit unstructured language tasks.