Session 9: Choosing a Prediction Model — Cross-Validation, Lasso and Ridge
Monday, 23 November 2026, 09:00–11:15 · Room MAD 099
What this session is about
We predict the prices of houses sold in Ames, Iowa. Each sale is described by more than seventy features — size, age, neighbourhood, basement, garage, roof — and there is no theory about which of them matter for the price. We fit our models on a deliberately small sample of 300 sales: more than 200 candidate predictors for 300 observations, exactly the situation in which ordinary regression breaks down. A separate holdout of sales is set aside before any model is fitted, to judge the final predictions once.
Session 8 ended with two problems. Today solves one in each part.
- Part A — Cross-validation. One train/test split is luck. Splitting five times and averaging is not. Cross-validation chooses among models inside the fitting sample; the holdout only judges, once, at the end.
- Part B — Lasso and ridge. With hundreds of candidate predictors and few observations, ordinary least squares overfits badly. The lasso adds a penalty that shrinks coefficients and sets many of them to exactly zero — automatic variable selection, with the strength of the penalty chosen by cross-validation. Ridge regression shrinks all coefficients instead. Both predict better than a model chosen by hand — and we see when that advantage disappears.
Before any model, real data needs preparing: a zero that really means “not recorded” is replaced by the median — computed without the holdout — with a flag that says it was missing; the data become a numeric matrix for glmnet; and columns that only a handful of houses have are dropped.
One new package. Install it before the session:
install.packages("glmnet")Learning goals
After this session you can:
- Explain why a single train/test split is not enough to choose a model
- Run k-fold cross-validation by hand and read its average and spread
- Explain why choosing a model and judging it need different data, and why a holdout set is used only once
- Spot a missing value disguised as a number, fill it in with the median and a missing-value flag, and say why the flag matters
- Turn a data frame into a numeric matrix with
model.matrix() - Explain what the lasso penalty does, read a coefficient path, and choose λ with
cv.glmnet()—lambda.minand the 1-SE rule (lambda.1se) - Say how ridge differs from the lasso, and why its squared penalty shrinks coefficients without setting them to zero
- Say when penalised regression pays off — many predictors per observation — and when it does not
- Read the predictors a lasso keeps as good predictors — not as causes
Slides
Materials
| File | Description |
|---|---|
| Exercise sheet | The in-session exercise |
| ames.csv | House sales in Ames, Iowa |
| airbnb-hackney.csv | Airbnb listings in Hackney, London — optional exercise |
| Slides | Online after the session |
| Live coding document | Online after the session |
| Exemplary solution | Online after the session |
Data
House sales in Ames, Iowa, 2006 to 2010: one row per sale, with the price in US dollars and 73 features of the house (De Cock 2011). The file is an unmodified export of the version in the R package modeldata, in which a missing lot frontage is stored as 0. Every change — the flag, the dummies, the dropped rare columns — is made in the code. The MIT licence covers the modeldata package; the data themselves come from the Ames City Assessor’s Office and were compiled by De Cock for teaching.
The optional exercise uses the Airbnb listings in Hackney, London, March 2017, from Békés and Kézdi (2021a): the file airbnb_hackney_workfile of the book’s data repository (Békés and Kézdi 2021b), downloaded on 12 October 2026, licensed CC BY-NC-SA 4.0. Unmodified apart from the file name; all changes are made in the code.
The question we answer
How do you choose among hundreds of possible models — without fooling yourself?
Choose on data the model has not seen, many times over: cross-validation. Let a penalty decide which predictors earn their place: the lasso, or shrink them all: ridge. And keep one set of houses out of every choice, to say at the end, once, how wrong the chosen model will be.
With 300 sales and over 200 candidates, a regression with everything is worse than eight predictors picked by an expert — and lasso and ridge beat the expert without knowing anything about houses. With eight times as many sales, the advantage almost disappears. Penalties are insurance: worth most when predictors are many and observations few.
Take-home until session 10
Read James et al. (2021), ch. 5.1 “Cross-Validation” and ch. 6.2 “Shrinkage Methods”, free at statlearning.com. The reasoning behind today’s two parts, with the mathematics kept light.
Session 10 leaves prediction behind: no outcome at all, only customers to group — clustering for segmentation. Report 3 is issued.
Going further
- James et al. (2021), ch. 6.1 — subset selection, the older alternative to the lasso
- Békés and Kézdi (2021a), ch. 14 — predicting Airbnb prices, with the London data of the optional exercise
- The live coding document has an optional appendix that does today’s work with the
tidymodelsframework