Module 3: Linear regression: the first model · Lesson 3.5 · about 25 min
Measuring fit: residuals, RMSE and R²
How good is a regression model? RMSE and MAE give the typical error in real units, R² gives the share of variation explained, and a residual plot shows what the scores hide. Switch between four datasets and read the residuals.
Module 3 · about 25 minutes · read first: 3.2. You’ll be able to compute MAE, RMSE and R², choose the right one for a job, and diagnose a model from its residual plot.
1. Why this matters
A model always produces numbers, even a bad one, just as a broken clock always shows a time. Before anyone acts on its predictions they need two answers: how far off is it, typically, and is it missing something systematic? The scores answer the first. The residual plot answers the second, and catches problems the scores hide.
2. Three scores
With residuals on test rows:
| Score | Formula | Units | Reads as |
|---|---|---|---|
| MAE (mean absolute error) | same as y | “on average we are off by this much” | |
| RMSE (root mean squared error) | same as y | “a typical error”, with big misses counted more | |
| R² (coefficient of determination) | none | “share of the variation the model explains” |
Choose by what hurts: if one miss of 10 is much worse than five misses of 2 (stock levels, dosing), use RMSE. If every unit of error costs the same (delivery minutes), MAE is more honest. R² is good for comparing models on the same data, but it says nothing about whether the error is small enough for the job: an R² of 0.9 can still mean errors of $50,000 on house prices.
Always quote scores on test data. On training data they flatter.
Here are all three scores for one small test, step by step. True sandwich sales on four days were 3, 5, 7 and 9 (dozens); a model predicted 4, 5, 6 and 9:
3. Reading residuals
Plot each residual against its prediction. A good model leaves residuals that look like random noise: centred on zero, the same spread everywhere, no shape.
| You see | It means | Try |
|---|---|---|
| Random cloud around 0 | The model has caught the pattern | Nothing; it is as good as this feature set allows |
| A curve (U or arch) | The relationship bends | Squared terms (Lesson 3.6), a log, or a tree model |
| A funnel (spread grows) | Errors grow with the size of y | Predict log(y), or report errors in percent |
| One point far out | An outlier or a data error | Check the row; consider capping or a robust loss |
| Groups or stripes | A missing category | Add the feature that separates the groups |
Maths and principle
Principle: R² compares the model with the simplest baseline, always predicting the mean .
Equations:
SSE is the model’s sum of squared errors; SST (“total”) is the baseline’s. On the training data of a least-squares line, R² equals the squared correlation (Lesson 2.3). On test data R² can be negative: the model is worse than the mean.
Also, , so lowering one lowers the other, and always.
Analogy: the baseline is a guess with no information. R² asks how much of the guess’s error the model removes.
Try it: y = 3, 5, 7, 9; predictions 4, 5, 6, 9. Find R². (; SST = 9 + 1 + 1 + 9 = 20; SSE = 1 + 0 + 1 + 0 = 2; R² = 1 − 2/20 = 0.9.)
- MAE
- The average size of a miss, in the units of y. Every unit of error counts the same.
- RMSE
- The square root of the average squared miss. Big misses count extra.
- SSE and SST
- The model's total squared error, and the total squared error of always guessing the mean.
- R² = 1 − SSE ÷ SST
- The share of the baseline's error the model removes: 1 is perfect, 0 is no better than the mean, below 0 is worse.
- Physical meaning
- MAE is what an average customer feels: "it's usually about 9 minutes late". RMSE is what the complaints desk feels, because the one order that was 40 minutes late hurts far more than four that were 10 minutes late. R² is what the manager asks: "how much better is this than just guessing the usual?".
“An R² of 0.9 means the predictions are good enough.” R² compares with the mean, not with what the job needs. House prices that vary by $500,000 can give R² = 0.9 while the typical miss is still $100,000, too much for a buyer. Always quote an error in real units (MAE or RMSE) next to R².
4. Worked example
Made-up numbers. Two models predict delivery times (minutes) for the same 6 test orders, whose true times are 20, 25, 30, 35, 40, 90 (the last was a road closure).
| errors | MAE | RMSE | |
|---|---|---|---|
| Model A | 3, −3, 3, −3, 3, −40 | 9.2 | 16.6 |
| Model B | 8, −8, 8, −8, 8, −20 | 10.0 | 11.0 |
Model A is closer on five of six orders and has the lower MAE; Model B has the lower RMSE because it misses the road closure by less. Which is better? If a 40-minute surprise loses a customer, the business should prefer B and RMSE. If every minute costs the same, A and MAE.
(The arithmetic for A: MAE = (3×5 + 40)/6 = 55/6 ≈ 9.2. RMSE = √((9×5 + 1600)/6) = √274.2 ≈ 16.6.)
5. Practice
- Errors of 1, −1, 1, −1. Find MAE and RMSE.
- A model’s test R² is −0.2. What does that tell you?
- A residual plot fans out to the right. Name one fix.
Answers
- Both are 1.
- It is worse than predicting the mean for everyone: back to the drawing board (often a sign of overfitting or a bug).
- Model log(y) instead of y, or judge errors in percent.
Try it on your own data: the regression workbench shows R² and a residual plot for any model you fit.
New words: MAE, RMSE, R², SSE and SST, residual plot, heteroscedasticity (a funnel: the error spread changes with the prediction).
- MAE is the average size of the errors; RMSE counts big errors more; both are in the units of y, and MAE ≤ RMSE.
- R² = 1 − SSE ÷ SST is the share of the mean-guess error the model removes; on test data it can be negative.
- Choose the score by what hurts the business, and always quote it on test data.
- A residual plot should look like random noise; curves, funnels and groups point to what the model is missing.
Check yourself
Sources
- James, G., Witten, D., Hastie, T., Tibshirani, R. and Taylor, J. (2023), An Introduction to Statistical Learning with Applications in Python, Springer, Sections 2.2 and 3.1 (assessing model accuracy, R²) https://www.statlearning.com/
- scikit-learn user guide: Regression metrics https://scikit-learn.org/stable/modules/model_evaluation.html
- Hyndman, R. J. and Athanasopoulos, G. (2021), Forecasting: Principles and Practice, 3rd ed., OTexts, Section 5.8 (evaluating point forecast accuracy) https://otexts.com/fpp3/
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. Numbers in the examples are made up to show the idea unless a source is given. Simulations run in your browser; nothing you type is sent anywhere.