Session 11: Supervised Learning — Classification
Monday, 7 December 2026, 09:00–11:15 · Room MAD 099
What this session is about
A bank lends to small Hungarian firms — mostly restaurants and hotels — and wants to know before it lends: will this firm still be in business next year? The outcome is a yes or a no. Predicting it is an instance of classification: the part of machine learning that is about predicting discrete rather than continuous outcomes.
More specifically, in this session we will cover:
- Many ways to say yes or no. An overview of classification methods — logistic regression, k nearest neighbours, decision trees, random forests, boosting, neural networks — and what each is used for. Then a short introduction to the one that will feel familiar: logistic regression, session 8’s regression adapted to predicting probabilities.
- Trees and forests. The focus of the session. A decision tree predicts with a flowchart of yes/no questions that anyone can read; grown too far it memorises the data, so its size is chosen by cross-validation. A random forest lets hundreds of trees vote — usually more accurate, but a black box.
- Decisions. Turning a probability into a decision needs a threshold. We discuss how to choose one and which criteria judge the result. Above all, we see why a model that is “94% accurate” can be useless, and how the confusion matrix, sensitivity and specificity, the ROC curve and the AUC help to make good decisions. One decision no computer can take for you: what the two kinds of mistakes cost.
Three new packages — install them before the session:
install.packages(c("rpart.plot", "ranger", "pROC"))rpart, for decision trees, comes with R.
Learning goals
After this session you can:
- Name common classification methods and what they are used for
- Fit a logistic regression with
glm()and read its predicted probabilities - Grow, prune and read a decision tree with
rpart(), choosing its size by cross-validation - Explain why a tree grown too far overfits, and how a random forest averages many trees; read its variable importance
- Explain why accuracy is misleading when one outcome is rare
- Build a confusion matrix and compute sensitivity and specificity
- Read an ROC curve and the AUC, and compare models by cross-validation
- Choose a classification threshold from the costs of the two kinds of mistakes
Slides
Materials
| File | Description |
|---|---|
| Exercise sheet | The in-session exercise |
| bisnode-firms.csv | Hungarian firms in 2012, and whether they exited (live coding) |
| telco-customers.csv | Customers of a telecom company, as in session 10 (exercise) |
| Slides | Online after the session |
| Live coding document | Online after the session |
| Exemplary solution | Online after the session |
Data
The live coding uses firms, the exercise the telecom customers from session 10.
Hungarian firms, one row per firm observed in 2012: log sales, log employees, profit margin, age, liquidity, foreign ownership, a female CEO, location in Budapest, industry — and whether the firm exited, i.e. was no longer in the data one year later. Most firms are restaurants or in accommodation.
The data comes from Békés and Kézdi (2021a). The file was built from the firm panel cs_bisnode_panel.csv in the book’s data repository (Békés and Kézdi 2021b), downloaded on 12 October 2026, licensed CC BY-NC-SA 4.0.
The panel has one row per firm and year, 2005 to 2016. The script prepare_bisnode.R turns it into one row per firm:
- Keep the firms observed in 2012.
- Exit = 1 if the firm does not appear in the 2013 data. This counts formal closures and firms that simply stopped filing accounts. (The book’s own case study, ch. 17, looks two years ahead; we look one.)
- Derive the variables: log sales and log employees (employees are annual averages, so they can be below one), the profit margin capped at −1 and 1, age = 2012 − founding year, liquidity = current assets / current liabilities, and 0/1 indicators for foreign ownership, a female CEO and a location in Budapest.
- Drop firms without valid sales, employees or profit margin, and with implausible ages.
The result: 17,506 firms, 11 columns. The raw panel is not in the repository — it is large. To run the script yourself, download the panel from the data repository, put it next to the script, and adjust the here::i_am() line at the top to where the script sits in your project — the path in the file is the one in the course repository.
Telecom customers, the file from session 10: one row per customer, with tenure, monthly and total charges, the services they use, their contract and payment method — and whether they left (churn). It is IBM’s fictional telecom sample dataset, distributed with the R package modeldata as wa_churn — the package is MIT-licensed, the data are IBM’s teaching sample, as in session 10.
The question we answer
When is a 94% accurate model useless?
When 94% of the cases belong to one group — such as firms that do not exit — a model that never predicts the rare outcome is 94% accurate, and so is a rule that ignores the data. Such a “model” hardly deserves the name and is clearly useless.
What matters is what the mistakes cost: a missed exit costs a bank far more than a good firm turned away. Judged that way, the best rules here flag any firm with an exit probability above about 10% — and are right only three times in four. Lower accuracy, better decision. And the random forest? It ranks firms only a little better than logistic regression: a flexible model helps only as much as the data allow.
Take-home until session 12
Read James et al. (2021), ch. 8.1 “The Basics of Decision Trees” and 8.2.2 “Random Forests”, free at statlearning.com. For logistic regression, the confusion matrix and the ROC curve: ch. 4.3 and the part of ch. 4.4.2 on them. A general discussion of logistic regression, without the focus on prediction, but more beginner-friendly R implementation: Gräbner-Radkowitsch (2026). Also recommended: Békés and Kézdi (2021a), ch. 17 — the same firm data and a very similar question, with random forests.
Session 12: recap, synthesis and exam preparation.
Going further
- Gräbner-Radkowitsch (2026) — logistic regression in R, step by step
- James et al. (2021), ch. 8.2.3 — boosting, the method that often beats random forests on tables of data
- The
pROCpackage — ROC curves, AUC with confidence intervals, comparing two curves