Module 2 of 6 · 30 minutes

Regression: predicting numbers

From sales forecasts to delivery times, many business questions are 'how much?' This module covers the workhorse model for those questions.

Learning objectives

Explain linear regression
Train a regression model and compare with a mean baseline
Interpret MAE, RMSE and R²
Read coefficients carefully
Inspect errors visually

2.1 Linear regression: drawing the best line

Linear regression predicts a number as a weighted sum of the features plus a constant. Training finds the weights that minimise the squared differences between predictions and actual values. It's fast, interpretable and a strong first model for regression.

Linear regression draws the line that minimises the squared residuals — the vertical gaps between each point and the line.

2.2 Train and evaluate

Python · predicting annual customer valuefrom sklearn.linear_model import LinearRegression
from sklearn.dummy import DummyRegressor
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

reg_features = ["tenure_months", "monthly_spend", "support_tickets", "on_promo"]
Xr, yr = customers[reg_features], customers["annual_value"]
Xr_train, Xr_test, yr_train, yr_test = train_test_split(Xr, yr, test_size=0.2, random_state=0)

base = DummyRegressor(strategy="mean").fit(Xr_train, yr_train)
lin = LinearRegression().fit(Xr_train, yr_train)

for name, m in [("Baseline (mean)", base), ("Linear regression", lin)]:
    pred = m.predict(Xr_test)
    mae = mean_absolute_error(yr_test, pred)
    rmse = mean_squared_error(yr_test, pred) ** 0.5
    print(f"{name:18}  MAE £{mae:6.1f}   RMSE £{rmse:6.1f}   R² {r2_score(yr_test, pred):.2f}")

2.3 Regression metrics

Metric	Meaning	Use when
MAE (mean absolute error)	Average size of errors, in the target's units	Easy to explain: "off by £40 on average"
RMSE (root mean squared error)	Like MAE but penalises big errors more	Large errors are especially costly
R²	Share of the variation explained, compared with predicting the mean (1 is perfect, 0 is no better than the mean)	Comparing models on the same data

2.4 Reading the coefficients

coefs = pd.Series(lin.coef_, index=reg_features).round(2)
print(coefs)
# e.g. monthly_spend ≈ +10: each extra £1 of monthly spend adds about £10 of annual value

Coefficients show how the prediction changes when one feature rises by one unit, holding the others constant. They're easy to explain — but they describe association, not causation, and become unreliable when features are strongly correlated with each other.

2.5 Check the errors, not just the score

import matplotlib.pyplot as plt

pred = lin.predict(Xr_test)
plt.scatter(yr_test, pred, s=8, alpha=0.5)
plt.plot([yr_test.min(), yr_test.max()], [yr_test.min(), yr_test.max()], "r--")
plt.xlabel("Actual annual value (£)"); plt.ylabel("Predicted (£)")
plt.show()

Points close to the diagonal are good predictions. Patterns — such as consistent under-prediction for high-value customers — suggest missing features or a non-linear relationship.

Exercise: add a feature

Add region to the regression by one-hot encoding it (e.g. with pd.get_dummies(..., drop_first=True)). Does MAE improve? Why or why not?

Show a solution

Xr2 = pd.get_dummies(customers[reg_features + ["region"]], columns=["region"], drop_first=True)
Xa, Xb, ya, yb = train_test_split(Xr2, yr, test_size=0.2, random_state=0)
m = LinearRegression().fit(Xa, ya)
print(mean_absolute_error(yb, m.predict(Xb)))
# Little or no improvement: region played no part in how this synthetic data was generated.

Key takeaways

Linear regression fits a weighted sum of features.
MAE is the average error in real units; RMSE punishes big errors; R² compares with predicting the mean.
Coefficients show association, not causation.
Plot predicted vs actual to find systematic errors.