Show code
# The Python chunks on this page import pandas; reticulate provisions it.
reticulate::py_require(c("pandas", "scikit-learn", "scipy", "matplotlib"))Inputs, outputs, kinds of problems and learning from experience
Last modified: 2026-11-19
# The Python chunks on this page import pandas; reticulate provisions it.
reticulate::py_require(c("pandas", "scikit-learn", "scipy", "matplotlib"))Course Overview: What Is Machine Learning? once held the material below as well. It keeps the definition of machine learning, why to learn from data, where machine learning fits, and the course ahead. Both chapters use the Auto and spam data.
A machine learning system works in two steps, shown in Figure 1. In the first step, training (Definition 3), a learning algorithm (Definition 2) reads some training data \(\trainset\) (Definition 1) and produces a model (Definition 5), a function \(\fitmodf{\trainset}\). Training happens once, ahead of time (train time). In the second step, prediction, the system feeds a new input \(x\) to the model and gets back a prediction \(\fitmodf{\trainset}\paren{x}\). Prediction happens every time the system is used (run time). The next section defines each of these terms.
Exercise 1 (What are the parts of a learning problem?) Figure 1 shows training and then prediction. Name the three objects that training involves:
Solution 1. Training involves three objects:
Definition 1 (Training data (training set)) The training data are inputs \(x\), possibly with outputs \(y\).
Other names: the training data are also called the training set.
Definition 2 (Learning algorithm) A learning algorithm turns the training data (Definition 1) into a function \(h\).
Definition 3 (Training (fitting, learning, estimation)) Running a learning algorithm (Definition 2) on the training data is called training.
Other names: training is also called fitting, learning or estimation.
Definition 4 (Feature (attribute)) A feature is one number that a model family receives as input [1]: one entry of the output of the preprocessing applied to an input’s covariates. With no preprocessing, the features are the covariates.
Other names: a feature is also called an attribute. A covariate, a predictor, an input variable and an independent variable are the measured values before preprocessing.
Remark 1 (A model represents a phenomenon). In science, a model represents an object, a phenomenon or a process, and it makes some part of the world easier to understand, quantify or simulate [2], [3]. The candidates in a model represent how the outcomes of a data set depend on its inputs. Choosing the set of candidates is an assumption: the set rules out every description outside it. George Box summed up what that assumption costs as “all models are wrong but some are useful” [4, p. 202]. He argued that no simple model represents a system in the world exactly, but a well-chosen simple model can still approximate it usefully, as the ideal gas law \(PV = RT\) does for actual gases, none of which obeys it exactly [4, pp. 202–203]. So the question to ask of a model is not whether it is true. In his words, the question is “Is the model illuminating and useful?” [4, p. 203]. Box also noted that a method derived without an explicitly stated model still implies some unstated model [4, p. 201], so even a method that only makes predictions rests on assumptions about the data. A conceptual model is the same idea outside mathematics: a representation of a system, built from concepts, that helps people understand it.
Definition 6 (Statistical model) A statistical model is a model whose candidates are distribution models: either a distribution model family or a fitted distribution model.
Definition 7 (Predictive model) A predictive model is a model whose candidates are the members of a prediction function family: either the family itself or one member of the family that training has chosen.
Example 1 (Gaussian outcomes with an affine mean: a model of both kinds) Take the set of distribution models that give an outcome \(\Outvar\), given a covariate value \(\predvar\), a Gaussian distribution whose mean is an affine function of \(\predvar\). That set is a statistical model. Each member also gives a mean function, and that function predicts the outcome at each covariate value. So the same assumptions also make a predictive model. Figure 2 draws one member.
intercept <- 1
slope <- 0.5
noise_sd <- 1
x_at <- c(2, 5, 8)
y_grid <- seq(-3, 3, length.out = 101)
curves <- tidyr::expand_grid(x_at = x_at, dev = y_grid)
curves <- dplyr::mutate(
curves,
y = intercept + slope * x_at + dev,
density = dnorm(dev, sd = noise_sd),
x = x_at + 2 * density
)
x_limits <- c(0, 10)
y_limits <- c(-2, 9)
curves |>
ggplot2::ggplot() +
ggplot2::aes(x = x, y = y, group = x_at) +
ggplot2::geom_abline(
intercept = intercept,
slope = slope,
colour = "#0072B2"
) +
ggplot2::geom_path(colour = "#D55E00") +
ggplot2::coord_cartesian(xlim = x_limits, ylim = y_limits) +
ggplot2::labs(x = "covariate value x", y = "outcome y")import matplotlib.pyplot as plt
import numpy as np
intercept, slope, noise_sd = 1, 0.5, 1
dev = np.linspace(-3, 3, 101)
density = np.exp(-0.5 * (dev / noise_sd) ** 2) / (noise_sd * np.sqrt(2 * np.pi))
x_line = np.array([0, 10])
fig, ax = plt.subplots(figsize=(5, 3.5), layout="constrained")
ax.plot(x_line, intercept + slope * x_line, color="#0072B2");
for x_at in [2, 5, 8]:
ax.plot(x_at + 2 * density, intercept + slope * x_at + dev, color="#D55E00");
ax.set_xlim(0, 10);
ax.set_ylim(-2, 9);
ax.set_xlabel("covariate value x");
ax.set_ylabel("outcome y");
plt.show()Remark 2 (Causal models are a third kind). A causal model is a further kind of model. Its candidates also describe how the distribution of the data would change under an intervention, such as setting one variable to a chosen value. So a causal model carries a statistical model with it, but a statistical model says nothing about interventions. These notes do not use causal models.
Remark 3 (Which meaning of “model” these notes use). A use of the word model names either a set of candidates or one chosen member, and the surrounding sentence shows which one. These notes usually use the word for a predictive model that training has chosen: the function \(\fitmodf{\trainset}\) that training produces from the training data \(\trainset\) and a prediction function family. The chosen function turns a new input value \(\newpredval\) into the prediction \(\fitmodvalf{\trainset}{\newpredval}\). The input value \(\newpredval\) holds one value of each feature. The hat in \(\fitmodf{\trainset}\) marks a function estimated from data.
Remark 4 (“Predictor” can name a model). Statistics uses predictor for a measured value of an input, a covariate. Some machine learning texts use the same word for a fitted model. Hastie, Tibshirani and Friedman call the averaged forest function “the random forest (regression) predictor” [5], equation 15.2. Murphy calls the function obtained by training on a subset of the data “the predictor” [6], equation 16.104. These notes use predictor only for a measured value of an input.
Example 2 (The parts of the spam cutoff problem) Name the parts of the spam cutoff rule on the Spambase data [7]: the training data (Definition 1) are the \((x, y)\) pairs of half the messages, the learning algorithm (Definition 2) is a function that searches cutoffs, training (Definition 3) is running it, and the model (Definition 5) is the rule it returns.
import numpy as np
import pandas as pd
spam_tp = pd.read_csv("data/spambase.csv")
n_tp = len(spam_tp)
order_tp = (np.arange(n_tp) * 1009) % n_tp # scrambled row order, as before
train_tp, new_tp = order_tp[: n_tp // 2], order_tp[n_tp // 2 :]
# Training data: x is the share of "!" characters, y the label (1 = spam).
x_tp = spam_tp["char_freq_!"].to_numpy()
y_tp = spam_tp["Class"].to_numpy()
def learn_cutoff_tp(x, y):
# The learning algorithm: turn training data into a cutoff.
grid = np.round(np.arange(0, 3.01, 0.05), 2)
return grid[np.argmin([np.mean((x > c) != (y == 1)) for c in grid])]
cut_tp = learn_cutoff_tp(x_tp[train_tp], y_tp[train_tp]) # training
def model_tp(x):
# The model: map a new input x to a prediction.
return (x > cut_tp).astype(int)
print("cutoff:", cut_tp)cutoff: 0.1
x prediction label
0 1.103 1 1
1 0.000 0 0
2 0.000 0 0
3 0.633 1 1
4 0.443 1 1
spam_tp <- read.csv("data/spambase.csv", check.names = FALSE)
n_tp <- nrow(spam_tp)
order_tp <- ((seq_len(n_tp) - 1) * 1009) %% n_tp + 1 # scrambled row order, as before
train_tp <- order_tp[seq_len(n_tp %/% 2)]
new_tp <- order_tp[-seq_len(n_tp %/% 2)]
# Training data: x is the share of "!" characters, y the label (1 = spam).
x_tp <- spam_tp[["char_freq_!"]]
y_tp <- spam_tp[["Class"]]
learn_cutoff_tp <- function(x, y) {
# The learning algorithm: turn training data into a cutoff.
grid <- round(seq(0, 3, by = 0.05), 2)
grid[which.min(sapply(grid, function(c) mean((x > c) != (y == 1))))]
}
cut_tp <- learn_cutoff_tp(x_tp[train_tp], y_tp[train_tp]) # training
# The model: map a new input x to a prediction.
model_tp <- function(x) as.integer(x > cut_tp)
cat("cutoff:", cut_tp, "\n")cutoff: 0.1
five_tp <- new_tp[1:5]
data.frame(x = x_tp[five_tp], prediction = model_tp(x_tp[five_tp]), label = y_tp[five_tp])Training on 2300 messages produced the model “spam when ! makes up more than 0.1% of the characters”. The model never saw the five messages printed, and it gets 5 of them right. The same algorithm, trained on different data, can return a different model.
Abu-Mostafa names the components of a learning problem in his first Caltech lecture, from 3:06, and then formalizes them, from 12:47 [8], Lecture 1.
Exercise 2 (How can an email be an input?) A learning algorithm computes with numbers. How could you describe one email, or one car, so that a learning algorithm can use it as an input?
Solution 2. Measure a list of properties, and write each one as a number: for an email, how often each of several words appears; for a car, its horsepower, weight and number of cylinders.
Definition 8 (Feature vector) A feature vector \(x \in \reals^{\ninputvars}\) describes one input by \(\ninputvars\) features (Definition 4), one per measured property. It is an element of the real coordinate space \(\reals^{\ninputvars}\).
Example 3 (A column and a row of Spambase) In the Spambase table [7], each column except Class is a feature (Definition 4): word_freq_free is the percentage of a message’s words that are free. A message’s feature vector is its row of those columns.
0 0.32
1 0.14
2 0.06
Name: word_freq_free, dtype: float64
[0. 0.64 0.64 0. 0.32 0. ]
D = 57
[1] 0.32 0.14 0.06
# One feature vector: a row, with one value per feature.
x1_fv <- unlist(feats_fv[1, ], use.names = FALSE)
x1_fv[1:6][1] 0.00 0.64 0.64 0.00 0.32 0.00
D = 57
The feature word_freq_free has 4601 values, one per message. The first message’s feature vector has \(\ninputvars = 57\) entries, one per feature, so \(x \in \reals^{57}\) (Definition 8).
Weinberger introduces feature vectors, from 33:27 [9], Lecture 1.
Definition 9 (Feature function (feature map, encoding)) A feature function \(\vfeatmap\) sends a covariate vector \(\vpredvar\), whose entries may be numbers or categories, to its feature vector (Definition 8) \(\vfeatmap(\vpredvar) \in \reals^{\ninputvars \times 1}\).
Example 4 (Encoding a shelf location in the Carseats data) In the Carseats data [10], a store’s covariate vector holds its shelf location, a category (Bad, Medium or Good), and its price, a number. The feature function replaces the category with one 0/1 feature per category and keeps the price. The first three stores’ feature vectors are the rows below, so \(\ninputvars\) is the number of columns.
# One 0/1 column per shelf location, then the price.
data.frame(model.matrix(~ 0 + ShelveLoc + Price, data = carseats))Definition 10 (Output variable (label, target, response, outcome, dependent variable)) The output variable is the quantity a supervised model predicts for an input. Each example has one value of it, written \(\outvar\) once observed; before it is observed, it is a random variable, written \(\Outvar\). In regression the output is a number, and in classification it is a class.
Other names: the output is also called the label, target, response, outcome or dependent variable. The term outcome is common in statistics and the term target in machine learning.
Example 5 (The output of Spambase) In the Spambase table [7], the Class column is the output variable (Definition 10): it is 1 for a message that is spam and 0 for one that is not. The other columns are the features (Definition 4).
Of the 4601 messages, 1813 are spam, so the output takes the value 1 for 39.4% of them.
Most often, each input is a feature vector.
Exercise 3 (How long is the feature vector?) What is \(\ninputvars\) for each of these inputs?
An email in the spam data, described by the relative frequencies of 57 words and characters.
Solution 3. \(\ninputvars = 57\): one relative frequency per word or character.
Exercise 4 (How long is the feature vector? (part 2)) Use the setup of Exercise 3.
A handwritten ZIP-code digit, scanned as a 16-by-16 grid of grayscale pixels.
Solution 4. One intensity per pixel:
\[ \begin{aligned} \ninputvars &= 16 \times 16 \\ &= 256. \end{aligned} \]
Exercise 5 (Does every input fit in a row?) Consider four inputs:
Which of them arrive as a fixed list of attributes, and which do not?
Solution 5. Only the car arrives as a fixed list of attributes. Tabular data like it is structured; text, images and audio are unstructured.
Exercise 6 (Does every input fit in a row? (part 2)) Use the setup of Exercise 5.
For each one that does not, name a form in which a model could receive it.
Solution 6. An unstructured input can be presented to a model in several forms:
Definition 11 (Structured data) Data are structured when every observation is described by the same fixed list of attributes, so a dataset of them fits in a table with one row per observation and one column per attribute.
Example: a list of cars, each described by its engine size, weight and fuel economy.
Definition 12 (Unstructured data) Data are unstructured when they do not come as a fixed list of attributes: a model receives the raw content, such as words, pixels or audio samples, and any attributes have to be derived from it.
Examples: the text of an email, a photograph, a recorded sentence.
Example 6 (The numbers and the names in the Auto table) The Auto data [10] describe each car by the same fixed list of attributes: mpg, cylinders, displacement, horsepower, weight, acceleration, year and origin. The last column, name, is free text:
mpg cylinders displacement ... acceleration year origin
0 18.0 8 307.0 ... 12.0 70 1
1 15.0 8 350.0 ... 11.5 70 1
2 18.0 8 318.0 ... 11.0 70 1
[3 rows x 8 columns]
['chevrolet chevelle malibu', 'buick skylark 320', 'plymouth satellite']
name
ford 48
chevrolet 43
plymouth 31
Name: count, dtype: int64
[1] "chevrolet chevelle malibu" "buick skylark 320"
[3] "plymouth satellite"
make_sd
ford chevrolet plymouth
48 43 31
The numeric columns of the 392 cars are structured data (Definition 11): every row has the same 8 attributes. The name text is unstructured (Definition 12) until we derive an attribute from it, and the make we derived takes 37 distinct values. A derived attribute inherits the text’s quirks: chevrolet, chevy and chevroelt count as three makes.
An unstructured input can reach a model in several forms: flattened into a \(\ninputvars\)-vector, kept as a multi-dimensional array, given as a sequence, or structured as an object such as a parse tree; Solution 5 gives an example of each.
Historically, turning unstructured data (Definition 12) into good features was a large part of applied machine learning. Hand-engineered features create an information bottleneck: any attribute left out of the feature vector is lost to the model, and the choice of features reflects what its designers overlooked. Deep learning architectures, such as convolutional and recurrent networks, avoid the bottleneck by taking raw inputs and learning feature representations from them. Categorical inputs, such as a color, are turned into numbers too, for example by one-hot encoding (Harris’ 2022 weekly notes).
Plot.plot({
ariaLabel: 'Scatter plot of the Auto cars, with the x, y and color variables chosen from the controls; by default mpg against horsepower, colored by origin.',
height: 320, width: 760, grid: true,
x: {label: autoX}, y: {label: autoY},
color: {legend: true, label: autoColor,
type: autoColor === "origin" || autoColor === "cylinders" ? "ordinal" : "linear",
scheme: autoColor === "origin" || autoColor === "cylinders" ? "category10" : "viridis"},
marks: [Plot.dot(auto, {x: autoX, y: autoY, fill: autoColor, r: 3,
fillOpacity: 0.8, tip: true, title: "name"})]
})Auto variables plotted against each other, colored by a third.
In origin, 1 is American, 2 is European and 3 is Japanese.
Solution 7. There is no single right answer. Two natural choices:
mpg from horsepower, weight and the rest. Plotting mpg against horsepower (Figure 4) shows mpg falling as horsepower rises, and along a curve rather than a straight line.origin from the rest. The colors in Figure 4 show that the European and Japanese cars differ from the American ones in both mpg and horsepower.Exercise 8 (What differs between the two panels?) The two panels of Figure 5 show the same fifty points. What does the left panel give a learning algorithm that the right panel does not?
Solution 8. The left panel gives the group of every point, and the right panel gives only the points.
Exercise 9 (What differs between the two panels? (part 2)) Use the setup of Exercise 8.
What could the goal be in each of the two panels?
Solution 9. With the group of every point, the goal could be to predict the group of a new point. With only the points, the goal could be to find groups among them.
Definition 13 (Supervised learning) In supervised learning, every training example is a pair \((\vpredvar, \outvar)\): the correct output is given, and the goal is to predict \(\outvar\) for new \(\vpredvar\) [12].
Here \(\outvar\) is the output variable (Definition 10), which has other names there. A training example is also called an observation or an instance.
Definition 14 (Unsupervised learning) In unsupervised learning, the training data are inputs \(x\) alone, and the goal is to find structure in them, such as groups, directions of variation, or a density [13].
Example 7 (The Spambase table with and without its labels) Take three feature columns of Spambase [7], and the Class column that labels each message. With Class, the table holds \((x, y)\) pairs. Without it, the table holds inputs \(x\) alone.
import pandas as pd
spam_su = pd.read_csv("data/spambase.csv")
feats_su = ["word_freq_free", "word_freq_you", "char_freq_!"]
# Supervised: each row is a pair (x, y), and the goal is to predict y from x.
# Comparing the label groups shows which features carry information about y.
print(spam_su[feats_su + ["Class"]].head(3)) word_freq_free word_freq_you char_freq_! Class
0 0.32 1.93 0.778 1
1 0.14 3.47 0.372 1
2 0.06 1.36 0.276 1
word_freq_free word_freq_you char_freq_!
Class
0 0.07 1.27 0.11
1 0.52 2.26 0.51
word_freq_free word_freq_you char_freq_!
word_freq_free 1.00 0.08 0.10
word_freq_you 0.08 1.00 0.15
char_freq_! 0.10 0.15 1.00
spam_su <- read.csv("data/spambase.csv", check.names = FALSE)
feats_su <- c("word_freq_free", "word_freq_you", "char_freq_!")
# Supervised: each row is a pair (x, y), and the goal is to predict y from x.
# Comparing the label groups shows which features carry information about y.
head(spam_su[, c(feats_su, "Class")], 3) word_freq_free word_freq_you char_freq_!
word_freq_free 1.00 0.08 0.10
word_freq_you 0.08 1.00 0.15
char_freq_! 0.10 0.15 1.00
In the supervised view (Definition 13), spam averages 0.52% free against 0.07% in email, so free helps predict the label. In the unsupervised view (Definition 14) there is no label to predict: the only finding is structure in the inputs, such as a correlation of 0.08 between free and you.
Hastie and Tibshirani present the supervised learning problem, then unsupervised learning, from 4:02 [14], video 1.2.
GeeksforGeeks explains supervised and unsupervised learning [15].
Exercise 10 (What if labels are scarce, or there are none?) Suppose only a few of the training examples have labels. A harder case: no one gives the correct answer at all, and a game program learns only whether it won. What kind of learning does each case call for?
Solution 10. An algorithm given a few labels could learn from the labeled and the unlabeled examples together. A program given no correct answers could learn from the results of its own moves, counting a win as good and a loss as bad.
Definition 15 (Semi-supervised learning) In semi-supervised learning, the training data hold a few labeled examples and many unlabeled ones [16]. For a spam filter, the labeled examples are the emails someone has read and marked as spam or email; the rest of the inbox is unlabeled.
Example 8 (A spam filter with 100 labeled messages) Suppose only 100 of the Spambase messages [7] have been read and marked as spam or email, and the rest of the inbox is unlabeled.
import numpy as np
import pandas as pd
spam_ss = pd.read_csv("data/spambase.csv")
n_ss = len(spam_ss)
order_ss = (np.arange(n_ss) * 1009) % n_ss # scrambled row order, as before
# Hide every label except those of the first 100 messages in the scrambled order.
y_ss = np.array(spam_ss["Class"], dtype=float) # a writable copy
y_ss[order_ss[100:]] = np.nan
print("labeled:", int(np.sum(~np.isnan(y_ss))), " unlabeled:", int(np.sum(np.isnan(y_ss))))labeled: 100 unlabeled: 4501
# Every message still has its features; only the labeled ones can score a cutoff.
x_ss = spam_ss["char_freq_!"].to_numpy()
lab_ss = ~np.isnan(y_ss)
grid_ss = np.round(np.arange(0, 3.01, 0.05), 2)
errs_ss = [np.mean((x_ss[lab_ss] > c) != (y_ss[lab_ss] == 1)) for c in grid_ss]
cut_ss = grid_ss[np.argmin(errs_ss)]
# Check it against the labels we hid.
truth_ss = spam_ss["Class"].to_numpy()[~lab_ss]
print("cutoff:", cut_ss, " error on the unlabeled:",
round(np.mean((x_ss[~lab_ss] > cut_ss) != (truth_ss == 1)), 3))cutoff: 0.05 error on the unlabeled: 0.217
spam_ss <- read.csv("data/spambase.csv", check.names = FALSE)
n_ss <- nrow(spam_ss)
order_ss <- ((seq_len(n_ss) - 1) * 1009) %% n_ss + 1 # scrambled row order, as before
# Hide every label except those of the first 100 messages in the scrambled order.
y_ss <- spam_ss$Class
y_ss[order_ss[-seq_len(100)]] <- NA
missing_ss <- is.na(y_ss)
n_unlab_ss <- sum(missing_ss)
n_lab_ss <- length(y_ss) - n_unlab_ss
cat("labeled:", n_lab_ss, " unlabeled:", n_unlab_ss, "\n")labeled: 100 unlabeled: 4501
# Every message still has its features;
# only the labeled ones can score a cutoff.
x_ss <- spam_ss[["char_freq_!"]]
lab_ss <- !is.na(y_ss)
grid_ss <- round(seq(0, 3, by = 0.05), 2)
errs_ss <- sapply(grid_ss, function(c) mean((x_ss[lab_ss] > c) != (y_ss[lab_ss] == 1)))
cut_ss <- grid_ss[which.min(errs_ss)]
# Check it against the labels we hid.
truth_ss <- spam_ss$Class[!lab_ss]
cat("cutoff:", cut_ss, " error on the unlabeled:",
round(mean((x_ss[!lab_ss] > cut_ss) != (truth_ss == 1)), 3), "\n")cutoff: 0.05 error on the unlabeled: 0.217
The training data hold 100 labeled examples and 4501 unlabeled ones (Definition 15). This code uses only the labeled 100 to choose the cutoff, so the 4501 unlabeled messages go unused. A semi-supervised method would also learn from their features. The hidden labels were used above only to check the result.
Definition 16 (Reinforcement learning) In reinforcement learning, an agent acts and learns from rewards rather than from correct answers [17]. For a program learning to play a board game, for example, its actions are its moves, and its reward comes from how its games turn out, not from a label saying which move was right.
Example 9 (A filter that learns only from rewards) Let a spam filter act on the Spambase messages [7] one at a time: its action is to send each message to the junk folder or to the inbox, and the only feedback is a reward, +1 for a right call and -1 for a wrong one. Try three cutoff policies, each on its own 500 messages, and keep the one with the highest average reward:
import numpy as np
import pandas as pd
spam_rl = pd.read_csv("data/spambase.csv")
n_rl = len(spam_rl)
order_rl = (np.arange(n_rl) * 1009) % n_rl # scrambled message order, as before
x_rl = spam_rl["char_freq_!"].to_numpy()
y_rl = spam_rl["Class"].to_numpy()
def reward_rl(cut, rows):
# Act: junk the message if x > cut. The environment returns +1 or -1;
# the learner keeps these rewards, not the labels behind them.
return np.where((x_rl[rows] > cut) == (y_rl[rows] == 1), 1, -1)
for j, cut in enumerate([0.05, 0.5, 1.5]):
rows = order_rl[500 * j : 500 * (j + 1)]
print("cutoff", cut, "mean reward:", round(reward_rl(cut, rows).mean(), 3))cutoff 0.05 mean reward: 0.576
cutoff 0.5 mean reward: 0.432
cutoff 1.5 mean reward: 0.248
spam_rl <- read.csv("data/spambase.csv", check.names = FALSE)
n_rl <- nrow(spam_rl)
order_rl <- ((seq_len(n_rl) - 1) * 1009) %% n_rl + 1 # scrambled message order, as before
x_rl <- spam_rl[["char_freq_!"]]
y_rl <- spam_rl$Class
reward_rl <- function(cut, rows) {
# Act: junk the message if x > cut. The environment returns +1 or -1;
# the learner keeps these rewards, not the labels behind them.
ifelse((x_rl[rows] > cut) == (y_rl[rows] == 1), 1, -1)
}
cuts_rl <- c(0.05, 0.5, 1.5)
mean_reward_rl <- sapply(seq_along(cuts_rl), function(j) {
mean(reward_rl(cuts_rl[j], order_rl[(500 * (j - 1) + 1):(500 * j)]))
})
for (j in seq_along(cuts_rl)) {
cat("cutoff", cuts_rl[j], "mean reward:", round(mean_reward_rl[j], 3), "\n")
}cutoff 0.05 mean reward: 0.576
cutoff 0.5 mean reward: 0.432
cutoff 1.5 mean reward: 0.248
The policy with cutoff 0.05 earns the highest average reward, 0.58, so this agent would adopt it (Definition 16). This is the simplest case. In a game such as checkers, the reward arrives only when the game ends, and says nothing about which move was the mistake.
GeeksforGeeks explains:
Semi-supervised data are common, because labels cost time and money: someone has to read each email to mark it as spam, while unread emails arrive at no cost. The week 10 lecture on reinforcement learning replaces labels with rewards.
Exercise 11 (How could a computer find groups?) A dataset has points with no labels, and we want \(k\) groups. Propose a procedure that a computer could repeat to split the points into \(k\) groups.
Solution 11. Start with a rough guess at the groups, then keep improving the guess:
Definition 17 (Clustering) Clustering is the unsupervised task (Definition 14) of dividing \(N\) inputs \(\vx_1, \ldots, \vx_N\) into groups, called clusters, so that inputs in the same cluster are more similar to each other than to inputs in other clusters [20]. The output is a cluster assignment for each input. No labels are given, so the clusters are found, not learned from examples of the right answer.
Example 10 (Grouping cars without labels) Ignore mpg and every other column of the Auto data [10] except two: the horsepower and weight of each car. Standardize both, so that neither dominates the distances, and cluster the cars into three groups by two methods, \(k\)-means and hierarchical clustering with Ward linkage. Cluster labels are arbitrary, so number the clusters from the lightest cars to the heaviest:
import numpy as np
import pandas as pd
from scipy.cluster.hierarchy import fcluster, linkage
from sklearn.cluster import KMeans
auto_cl = pd.read_csv("data/auto.csv").dropna(subset=["horsepower"])
x_cl = auto_cl[["horsepower", "weight"]].to_numpy(float)
x_cl = (x_cl - x_cl.mean(axis=0)) / x_cl.std(axis=0, ddof=1)
w_cl = auto_cl["weight"].to_numpy(float)
def by_weight(labels, weight):
# Renumber the clusters 0, 1, 2 in order of their mean weight.
ids, inverse = np.unique(labels, return_inverse=True)
order = np.argsort([weight[labels == g].mean() for g in ids])
return np.argsort(order)[inverse]
km_cl = by_weight(KMeans(3, n_init=10, random_state=1).fit(x_cl).labels_, w_cl)
hc_cl = by_weight(fcluster(linkage(x_cl, "ward"), 3, criterion="maxclust"), w_cl)
print(np.bincount(km_cl), np.bincount(hc_cl))[159 138 95] [102 187 103]
col_0 0 1 2
row_0
0 102 57 0
1 0 130 8
2 0 0 95
auto_cl <- na.omit(read.csv("data/auto.csv")[, c("horsepower", "weight")])
x_cl <- scale(auto_cl)
# Renumber the clusters 1, 2, 3 in order of their mean weight.
by_weight <- function(labels, weight) {
rank(tapply(weight, labels, mean))[as.character(labels)]
}
set.seed(1)
km_fit <- kmeans(x_cl, centers = 3, nstart = 10)
km_cl <- by_weight(km_fit$cluster, auto_cl$weight)
hc_cl <- by_weight(cutree(hclust(dist(x_cl), "ward.D2"), k = 3), auto_cl$weight)
rbind(kmeans = table(km_cl), ward = table(hc_cl)) 1 2 3
kmeans 156 141 95
ward 102 187 103
table(kmeans = km_cl, ward = hc_cl) ward
kmeans 1 2 3
1 102 54 0
2 0 133 8
3 0 0 95
Both methods split the 392 cars into light, middle and heavy groups, and neither was told which cars are which. \(k\)-means puts 156, 141 and 95 cars in them, Ward linkage 102, 187 and 103. They agree on 330 cars and differ on the other 62. Each method is an answer to the same task, clustering (Definition 17). The two languages start \(k\)-means differently, so their counts can differ slightly. The prose gives the R values.
library(ggplot2)
cl_long <- rbind(
data.frame(auto_cl, method = "k-means", group = factor(km_cl)),
data.frame(auto_cl, method = "Ward linkage", group = factor(hc_cl))
)
cl_long$method <- factor(cl_long$method, levels = c("k-means", "Ward linkage"))
ggplot(cl_long, aes(horsepower, weight, colour = group, shape = group)) +
geom_point(alpha = 0.7) +
scale_colour_manual(values = c("1" = "#0072B2", "2" = "#D55E00", "3" = "#009E73")) +
scale_shape_manual(values = c("1" = 16, "2" = 17, "3" = 15)) +
facet_wrap(~method) +
labs(x = "Horsepower", y = "Weight (lb)", colour = "Group", shape = "Group")import matplotlib.pyplot as plt
fig, axes = plt.subplots(1, 2, figsize=(7, 3.5), sharey=True, layout="constrained")
styles_cl = [("#0072B2", "o"), ("#D55E00", "^"), ("#009E73", "s")]
for ax, labels, title in zip(axes, [km_cl, hc_cl], ["k-means", "Ward linkage"]):
for g, (colour, marker) in enumerate(styles_cl):
keep = labels == g
ax.plot(auto_cl["horsepower"][keep], auto_cl["weight"][keep], color=colour,
marker=marker, linestyle="", alpha=0.7, label=f"group {g + 1}");
ax.set_title(title);
ax.set_xlabel("Horsepower");
axes[0].set_ylabel("Weight (lb)");
fig.legend(*axes[0].get_legend_handles_labels(), loc="outside lower center", ncol=3);
plt.show()Remark 5 (Clustering is a task, not an algorithm). Clustering names the task, not a way of doing it. \(k\)-means (Definition 18) is one method for it, and hierarchical clustering is another, so the same inputs can be clustered in more than one way.
Definition 18 (k-means) k-means is a classic method for clustering (Definition 17) the inputs into \(k\) groups [21]. It starts with \(k\) group centers and repeats two steps until the assignments stop changing:
Example 11 (Two groups among the Spambase messages) Run \(k\)-means with \(k = 2\) on two Spambase features [7], the percentages of ! and $ characters, without showing it the labels. Both percentages are heavy-tailed, so the code works with \(\log(1 + \text{percentage})\), which keeps a few extreme messages from taking over:
import numpy as np
import pandas as pd
spam_km = pd.read_csv("data/spambase.csv")
feat_km = np.log1p(spam_km[["char_freq_!", "char_freq_$"]].to_numpy())
# Start from the messages with the smallest and the largest feature total.
sums_km = feat_km.sum(axis=1)
centers_km = feat_km[[sums_km.argmin(), sums_km.argmax()]]
for _ in range(100):
# Step 1: assign each point to its nearest center.
dist_km = ((feat_km[:, None, :] - centers_km[None, :, :]) ** 2).sum(axis=2)
group_km = dist_km.argmin(axis=1)
# Step 2: move each center to the mean of its points; stop when nothing moves.
new_km = np.array([feat_km[group_km == g].mean(axis=0) for g in range(2)])
if np.allclose(new_km, centers_km):
break
centers_km = new_km
print(centers_km.round(2))[[0.06 0.04]
[0.64 0.16]]
Class 0 1
group
0 2652 1025
1 136 788
spam_km <- read.csv("data/spambase.csv", check.names = FALSE)
feat_km <- log1p(as.matrix(spam_km[, c("char_freq_!", "char_freq_$")]))
# Start from the messages with the smallest and the largest feature total.
sums_km <- rowSums(feat_km)
start_km <- feat_km[c(which.min(sums_km), which.max(sums_km)), ]
# kmeans() repeats the assign and move steps; "Lloyd" is the version
# described in the definition.
fit_km <- kmeans(feat_km, centers = start_km, algorithm = "Lloyd", iter.max = 100)
round(fit_km$centers, 2) char_freq_! char_freq_$
1 0.06 0.04
2 0.64 0.16
# k-means never saw Class; compare its groups with it afterwards.
tab_km <- table(group = fit_km$cluster, Class = spam_km$Class)
tab_km Class
group 0 1
1 2652 1025
2 136 788
The groups hold 3677 and 924 messages. The method used only the features, as an unsupervised method does (Definition 14), yet 85% of the messages in group 2 are spam, against 28% in group 1. Groups found without labels can still line up with the labels.
library(ggplot2)
km_points <- data.frame(
excl = feat_km[, 1], dollar = feat_km[, 2],
group = factor(fit_km$cluster)
)
km_centers <- data.frame(excl = fit_km$centers[, 1], dollar = fit_km$centers[, 2])
ggplot(km_points, aes(excl, dollar, colour = group, shape = group)) +
geom_point(alpha = 0.4) +
geom_point(data = km_centers, shape = 4, size = 4, stroke = 1.5, colour = "black") +
scale_colour_manual(values = c("1" = "#0072B2", "2" = "#D55E00")) +
scale_shape_manual(values = c("1" = 16, "2" = 17)) +
labs(
x = "Log of one plus the percentage of !",
y = "Log of one plus the percentage of $",
colour = "Group", shape = "Group"
)import matplotlib.pyplot as plt
fig, ax = plt.subplots(figsize=(5, 4), layout="constrained")
for g, (colour, marker) in enumerate([("#0072B2", "o"), ("#D55E00", "^")]):
keep = group_km == g
ax.plot(feat_km[keep, 0], feat_km[keep, 1], color=colour, marker=marker,
linestyle="", alpha=0.4, label=f"group {g + 1}");
ax.plot(centers_km[:, 0], centers_km[:, 1], color="black", marker="x", markersize=10,
markeredgewidth=2, linestyle="", label="centre");
ax.set_xlabel("Log of one plus the percentage of !");
ax.set_ylabel("Log of one plus the percentage of $");
fig.legend(loc="outside lower center", ncol=3);
plt.show()GeeksforGeeks explains \(k\)-means clustering [22].
Exercise 12 (Let the computer find groups) Here are the cars of the Auto data again, plotted by weight and mpg, this time with no labels. First, by eye: how many groups do you see, and where are they?
kmZ = {
const mw = d3.mean(auto, (d) => d.weight), sw = d3.deviation(auto, (d) => d.weight);
const mm = d3.mean(auto, (d) => d.mpg), sm = d3.deviation(auto, (d) => d.mpg);
return auto.map((d) => [(d.weight - mw) / sw, (d.mpg - mm) / sm]);
}
// Lloyd's algorithm with a fixed start: the initial centers are the cars
// at evenly spaced weight quantiles, so every student sees the same run.
kmRun = (k, iters) => {
const order = d3.range(kmZ.length).sort((a, b) => auto[a].weight - auto[b].weight);
let centers = d3.range(k).map((i) => kmZ[order[Math.floor((i + 0.5) / k * kmZ.length)]]);
const assign = () => kmZ.map((p) => d3.minIndex(centers,
(c) => (p[0] - c[0]) ** 2 + (p[1] - c[1]) ** 2));
let labels = assign();
for (let t = 0; t < iters; t++) {
centers = d3.range(k).map((j) => {
const members = kmZ.filter((p, i) => labels[i] === j);
return [d3.mean(members, (p) => p[0]), d3.mean(members, (p) => p[1])];
});
labels = assign();
}
return labels;
}viewof kmShow = Inputs.radio(["no groups", "k-means groups"], {value: "no groups", label: "show"})
viewof kmK = Inputs.range([2, 5], {width: 360, value: 2, step: 1, label: "k (groups)"})
viewof kmIter = Inputs.range([0, 15], {width: 360, value: 0, step: 1, label: "iterations"})
viewof kmCompare = Inputs.radio(["nothing", "origin", "cylinders"], {value: "nothing", label: "compare with"})Plot.plot({
ariaLabel: 'Scatter plot of the Auto cars, mpg against weight. The points are gray until k-means groups are shown, and then each group has its own color.',
height: 320, width: 760, grid: true,
x: {label: "weight (lb)"}, y: {label: "mpg"},
color: {legend: kmShow === "k-means groups", type: "ordinal", scheme: "category10"},
marks: [Plot.dot(auto, {x: "weight", y: "mpg", r: 3, fillOpacity: 0.8,
fill: kmShow === "k-means groups" ? (d, i) => kmLabels[i] : "#777",
tip: true, title: "name"})]
}){
if (kmShow !== "k-means groups" || kmCompare === "nothing") return md``;
const values = Array.from(new Set(auto.map((d) => d[kmCompare]))).sort((a, b) => a - b);
const groups = Array.from(new Set(kmLabels)).sort();
const count = (g, v) => auto.filter((d, i) => kmLabels[i] === g && d[kmCompare] === v).length;
const head = `| ${kmCompare}: | ${values.join(" | ")} |`;
const rule = `|:--|${values.map(() => "--:").join("|")}|`;
const rows = groups.map((g) => `| ${g} | ${values.map((v) => count(g, v)).join(" | ")} |`);
return md`${[head, rule, ...rows].join("\n")}`;
}Auto cars’ weight and mpg, step by step, with its groups compared against origin or cylinders.
In origin, 1 is American, 2 is European and 3 is Japanese.
Solution 12. By eye, most people see two groups: light cars with high mpg, and heavy cars with low mpg.
Exercise 13 (Let the computer find groups (part 2)) Use the setup of Exercise 12.
Then let k-means (Definition 18) look. Choose \(k\), and step through the iterations. At what iteration does it stop changing?
Solution 13. K-means with \(k = 2\) agrees and stops changing after one iteration. It finds 217 light cars, averaging 2335 lb and 29 mpg, and 175 heavy ones, averaging 3774 lb and 17 mpg.Larger \(k\) also takes longer to stop changing: \(k = 3\) is still changing until its ninth iteration.
Exercise 14 (Let the computer find groups (part 3)) Use the setup of Exercise 12.
Then compare its groups with origin and with cylinders. Which one do the groups match better?
Solution 14. The groups match cylinders much better than origin. The heavy group holds all 103 eight-cylinder cars, and the light group holds 194 of the 199 four-cylinder cars. By origin, the heavy group is almost all American (163 of 175), but the light group mixes all three: 82 American, 58 European and 77 Japanese cars. European and Japanese cars have much the same weights and mpg, so no grouping of these two variables could separate them.
The mismatch with origin shows a limitation of unsupervised learning. The algorithm finds structure in the data, and that structure need not match the label we care about. The algorithm always returns \(k\) groups, even if the data hold fewer, so choosing \(k\) is itself a question. Unsupervised Learning returns to k-means, and to that question.
Exercise 15 (How do outputs differ?) Supervised problems are named for their output. Compare two supervised problems: predicting a house’s price, and labeling a message spam or email. How does the output \(y\) differ between them?
Solution 15. A house’s price is a number, and a message’s label is one of two categories.
Definition 19 (Regression) A supervised problem (Definition 13) is a regression problem when the output \(y\) is a quantity, such as a price or a temperature, or a vector of quantities [23].
Definition 20 (Classification) A supervised problem (Definition 13) is a classification problem when \(y\) is one of a fixed set of categories, such as spam or email [24].
With two classes the problem is binary classification; with more, it is multi-class classification.
Other names: the categories are also called classes.
Example 12 (Predicting mpg versus predicting spam) Look at the output \(y\) of two problems from this page: a car’s mpg in the Auto data [10], and a message’s Class in the Spambase data [7].
count 392.0
mean 23.4
std 7.8
min 9.0
25% 17.0
50% 22.8
75% 29.0
max 46.6
Name: mpg, dtype: float64
Class
0 2788
1 1813
Name: count, dtype: int64
Min. 1st Qu. Median Mean 3rd Qu. Max.
9.00 17.00 22.75 23.45 29.00 46.60
# Class takes just two values, standing for spam and email: a category.
table(spam_rc$Class)
0 1
2788 1813
Predicting mpg, which takes 127 distinct values between 9 and 46.6, is a regression problem (Definition 19). Predicting Class, which takes 2 values, is a classification problem (Definition 20), and a binary one because there are two classes. What decides the kind of problem is the output: predicting a car’s origin, coded 1, 2 or 3 but standing for a country, would be classification too.
library(ggplot2)
library(patchwork)
rg_mpg <- ggplot(auto_rc, aes(mpg)) +
geom_histogram(bins = 20, fill = "#0072B2", colour = "white") +
labs(x = "Miles per gallon", y = "Number of cars")
rg_class <- ggplot(
data.frame(Class = factor(spam_rc$Class, levels = 0:1, labels = c("email", "spam"))),
aes(Class, fill = Class)
) +
geom_bar(show.legend = FALSE) +
scale_fill_manual(values = c(email = "#0072B2", spam = "#D55E00")) +
labs(x = "Message class", y = "Number of messages")
rg_mpg + rg_classimport matplotlib.pyplot as plt
fig, (ax_mpg, ax_class) = plt.subplots(1, 2, figsize=(7, 3.5), layout="constrained")
ax_mpg.hist(auto_rc["mpg"], bins=20, color="#0072B2", edgecolor="white");
ax_mpg.set_xlabel("Miles per gallon");
ax_mpg.set_ylabel("Number of cars");
counts_rg = spam_rc["Class"].value_counts().sort_index()
ax_class.bar(["email", "spam"], counts_rg.to_numpy(), color=["#0072B2", "#D55E00"]);
ax_class.set_xlabel("Message class");
ax_class.set_ylabel("Number of messages");
plt.show()mpg of cars; on the right, the output of a classification takes just a few, such as the two Class values of messages (Example 12).
GeeksforGeeks explains classification and regression [25].
The spam filter is a classification problem with two classes. Predicting the log of a prostate-cancer antigen level in the blood from clinical measurements is a regression problem. So is predicting a house’s price from its living area, using 47 houses in Portland, Oregon. Other output types exist, such as a ranking of search results by relevance or a sequence of part-of-speech tags, one for each word of a sentence, but these two are the focus of most of this course.
Exercise 16 (Regression or classification?) Is predicting mpg from the other Auto data variables a regression or a classification problem?
Solution 16. Predicting mpg is regression, because mpg is a quantity.
Exercise 17 (Regression or classification? (part 2)) Use the setup of Exercise 16.
Is predicting origin a regression or a classification problem?
Solution 17. Predicting origin is classification, because origin is one of three categories, even though it is stored as the numbers 1, 2 and 3.
Exercise 18 (What is the simplest rule?) We want a rule that takes a car’s horsepower and returns a predicted mpg.
What is the simplest shape such a rule can take?
Exercise 19 (What is the simplest rule? (part 2)) Use the setup of Exercise 18.
How many numbers do we have to choose to specify that rule exactly?
Solution 19. Two: \(\hcoef{0}\) and \(\hcoef{1}\). To choose them, we need a way to score a line.
Exercise 20 (How could we score a line?) The line’s error on each car is \(y - \hat{y}\). Propose one number that scores the line on \(n\) cars, so that a lower score means a better line.
Solution 20. Square each car’s error, so that errors above and below the line both increase its total, and average the squares over the \(n\) cars.
Definition 21 (Sample mean squared error) The sample mean squared error (sMSE) of a fitted model on \(n\) examples \((x_1, y_1), \ldots, (x_n, y_n)\) is the average squared error [26], \[ \sMSEt = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2, \] where \(\hat{y}_i\) is the fitted model’s prediction for example \(i\).
Example 13 (A baseline’s sample mean squared error on Auto) The simplest rule for predicting mpg in the Auto data [10] ignores the car and always guesses the sample mean mpg. Compute its sMSE, and compare it with the sMSE of a line that uses horsepower:
import numpy as np
import pandas as pd
auto_ms = pd.read_csv("data/auto.csv")
y_ms = auto_ms["mpg"].to_numpy(float)
# Baseline: predict the mean mpg for every car.
mse_base_ms = np.mean((y_ms - y_ms.mean()) ** 2)
# Line: predict from horsepower, with the least squares fit.
slope_ms, intercept_ms = np.polyfit(auto_ms["horsepower"], y_ms, 1)
pred_ms = intercept_ms + slope_ms * auto_ms["horsepower"].to_numpy()
mse_line_ms = np.mean((y_ms - pred_ms) ** 2)
print("baseline sMSE:", round(mse_base_ms, 1))baseline sMSE: 60.8
line sMSE: 23.9
auto_ms <- read.csv("data/auto.csv")
y_ms <- auto_ms$mpg
# Baseline: predict the mean mpg for every car.
mse_base_ms <- mean((y_ms - mean(y_ms))^2)
# Line: predict from horsepower, with the least squares fit.
fit_ms <- lm(mpg ~ horsepower, data = auto_ms)
mse_line_ms <- mean((y_ms - fitted(fit_ms))^2)
cat("baseline sMSE:", round(mse_base_ms, 1), "\n")baseline sMSE: 60.8
line sMSE: 23.9
On these 392 cars the baseline’s sMSE (Definition 21) is 60.8, in squared miles per gallon, and the line’s is 23.9. The line’s sMSE cannot be larger, because the baseline is a line with slope 0. The line’s sMSE is smaller by a factor of 2.54.
Definition 22 (Sample root mean squared error) The sample root mean squared error (sRMSE) of a fitted model on \(n\) examples is the square root of its sMSE (Definition 21) [27], \[ \sRMSEt = \sqrt{\sMSEt}. \]
Remark 7 (The sRMSE has the unit of the outcome). Squaring the errors squares their unit, and the square root returns the sRMSE to the unit of the outcome. The sRMSE ranks fitted models in the same order as the sMSE (Definition 21), because the square root is increasing. We therefore report the sRMSE, and later say more about why.
Example 14 (A baseline’s sample root mean squared error on Auto) Take the square root of the two sMSEs from Example 13:
The baseline’s sRMSE (Definition 22) is 7.80 miles per gallon, the same unit as mpg, and the line’s is 4.89. Using horsepower shrinks the typical error by a factor of 1.59.
Hutchinson explains a model, training and a loss step by step, Machine Learning Example (21 min) [28].
GeeksforGeeks explains mean squared error [29].
Squaring makes every error increase the rule’s total, and makes big errors count much more than small ones.
Exercise 21 (Score a line by hand) Take the line \(\hat{y} = 40 - 0.16x\) and the first three cars in Auto:
| car | horsepower \(x\) |
mpg \(y\)
|
|---|---|---|
| chevrolet chevelle malibu | 130 | 18 |
| buick skylark 320 | 165 | 15 |
| plymouth satellite | 150 | 18 |
For each car, find the prediction \(\hat{y}\) and the error \(y - \hat{y}\). Then find the line’s sMSE (Definition 21) on these three cars, and take its square root to get the sRMSE (Definition 22).
Solution 21.
| car | \(\hat{y} = 40 - 0.16x\) | error \(y - \hat{y}\) | squared |
|---|---|---|---|
| chevrolet chevelle malibu | \(40 - 20.8 = 19.2\) | \(-1.2\) | 1.44 |
| buick skylark 320 | \(40 - 26.4 = 13.6\) | \(1.4\) | 1.96 |
| plymouth satellite | \(40 - 24.0 = 16.0\) | \(2.0\) | 4.00 |
The average of the squared errors is \[ \frac{1.44 + 1.96 + 4.00}{3} = \frac{7.40}{3} \approx 2.47. \] That average is the line’s sMSE on these three cars (Definition 21). Its square root, \(\sqrt{7.40/3} \approx 1.57\) miles per gallon, is the line’s sRMSE (Definition 22).
Exercise 22 (Fit a line yourself) Now score lines on all 392 cars at once. Move the intercept and slope to make the sRMSE (Definition 22) as small as you can. What is the lowest sRMSE you can reach?
autoFit = auto
mseOf = (predict) => d3.mean(autoFit, (d) => (d.mpg - predict(d.horsepower)) ** 2)
mpgMean = d3.mean(autoFit, (d) => d.mpg)
rmseOf = (predict) => Math.sqrt(mseOf(predict))
meanMse = mseOf(() => mpgMean)
// Least-squares fits, computed here rather than typed in,
// so the page cannot disagree with its own data.
lsLine = {
const xm = d3.mean(autoFit, (d) => d.horsepower);
const b1 = d3.sum(autoFit, (d) => (d.horsepower - xm) * (d.mpg - mpgMean)) /
d3.sum(autoFit, (d) => (d.horsepower - xm) ** 2);
return {b0: mpgMean - b1 * xm, b1};
}
lsQuad = {
// Solve the 3x3 normal equations for mpg ~ 1 + x + x^2.
const cols = autoFit.map((d) => [1, d.horsepower, d.horsepower ** 2]);
const A = [0, 1, 2].map((i) => [0, 1, 2].map((j) => d3.sum(cols, (c) => c[i] * c[j])));
const b = [0, 1, 2].map((i) => d3.sum(autoFit, (d, k) => cols[k][i] * d.mpg));
for (let p = 0; p < 3; p++) {
for (let r = p + 1; r < 3; r++) {
const f = A[r][p] / A[p][p];
for (let c = p; c < 3; c++) A[r][c] -= f * A[p][c];
b[r] -= f * b[p];
}
}
const w = [0, 0, 0];
for (let p = 2; p >= 0; p--) {
w[p] = (b[p] - d3.sum([0, 1, 2].filter((c) => c > p), (c) => A[p][c] * w[c])) / A[p][p];
}
return w;
}
hpGrid = d3.range(40, 236, 2)viewof fitB0 = Inputs.range([20, 60], {width: 360, value: 30, step: 0.1, label: "intercept b0"})
viewof fitB1 = Inputs.range([-0.4, 0.1], {width: 360, value: -0.05, step: 0.001, label: "slope b1"})
viewof fitShow = Inputs.checkbox(
["least-squares line", "least-squares curve"],
{label: "also show"}
){
const sign = (b) => (b < 0 ? "\u2212" : "+");
const ls = fitShow.includes("least-squares line")
? ` The least-squares line is mpg = ${lsLine.b0.toFixed(2)} ${sign(lsLine.b1)} ${Math.abs(lsLine.b1).toFixed(4)} \u00d7 horsepower, with sRMSE ${rmseOf((x) => lsLine.b0 + lsLine.b1 * x).toFixed(2)}.`
: "";
return md`Your line: mpg = ${fitB0.toFixed(1)} ${sign(fitB1)} ${Math.abs(fitB1).toFixed(3)} \u00d7 horsepower,
with sRMSE **${Math.sqrt(yourMse).toFixed(2)}**.
Predicting the sample mean, ${mpgMean.toFixed(1)}, for every car has sRMSE ${Math.sqrt(meanMse).toFixed(2)}.${ls}`;
}Plot.plot({
ariaLabel: 'Scatter plot of mpg against horsepower for the Auto cars, with a dotted gray line at the average mpg, a red line set by the intercept and slope controls, and optionally the least-squares line and curve.',
height: 340, width: 560, grid: true,
x: {label: "horsepower", domain: [40, 235]},
y: {label: "mpg", domain: [5, 50]},
marks: [
Plot.dot(autoFit, {x: "horsepower", y: "mpg", r: 2.5, fill: "#888", fillOpacity: 0.6,
tip: true, title: "name"}),
Plot.ruleY([mpgMean], {stroke: "#aaa", strokeDasharray: "2,3"}),
Plot.line(hpGrid, {x: (x) => x, y: (x) => fitB0 + fitB1 * x,
stroke: "#d62728", strokeWidth: 2.5, clip: true}),
fitShow.includes("least-squares line")
? Plot.line(hpGrid, {x: (x) => x, y: (x) => lsLine.b0 + lsLine.b1 * x,
stroke: "#1f77b4", strokeWidth: 2, strokeDasharray: "6,4", clip: true})
: null,
fitShow.includes("least-squares curve")
? Plot.line(hpGrid, {x: (x) => x, y: (x) => lsQuad[0] + lsQuad[1] * x + lsQuad[2] * x * x,
stroke: "#2ca02c", strokeWidth: 2, clip: true})
: null
]
})mpg against horsepower, with its mean squared error beside the least-squares fit’s.
The red line is yours; the dotted gray line is the average mpg.
Solution 22.
{
const ls = (x) => lsLine.b0 + lsLine.b1 * x;
const hand = (x) => 40 - 0.16 * x;
const lsRmse = rmseOf(ls);
const handRmse = rmseOf(hand);
const pctOver = 100 * (handRmse - lsRmse) / lsRmse;
const sign = lsLine.b1 < 0 ? "−" : "+";
const out = md`A line fitted by least squares has the lowest possible sRMSE, ${lsRmse.toFixed(2)} miles per gallon.
It is
$$
\\hat{y} = ${lsLine.b0.toFixed(2)} ${sign} ${Math.abs(lsLine.b1).toFixed(4)}x,
$$
the **least-squares** line (tick *least-squares line* to see it).
Each extra unit of horsepower predicts about ${Math.abs(lsLine.b1).toFixed(2)} fewer miles per gallon.
The line from @exr-mse-by-hand, $40 - 0.16x$, has an sRMSE of ${handRmse.toFixed(2)} on all 392 cars,
within ${pctOver.toFixed(1)}% of the best.
Nobody has to find the best line by sliding, though:
[linear regression](linear-regression.qmd) derives a [closed-form solution](https://morrison-lab.github.io/mds/expressions-algorithms.html#def-closed-form-solution) for it,
and [gradient descent](gradient-descent.qmd) finds it by repeatedly making small changes that lower the sMSE,
much as you just did by hand.`;
// OJS output is inserted into the DOM after this cell runs, and MathJax's
// own pass only fires once at page load -- so the $$...$$ this cell just
// wrote would never get typeset without an explicit, deferred retypeset.
requestAnimationFrame(() => window.Quarto?.typesetMath?.(out));
return out;
}Exercise 23 (Fit a line yourself (part 2)) Use the setup of Exercise 22.
How does the lowest sRMSE you reached compare with the sRMSE of ignoring horsepower and predicting the sample mean mpg for every car?
Solution 23.
Exercise 24 (Is a straight line the right shape?) Look at where the cars are relative to the least-squares line, at low, middle and high horsepower. Is there a pattern in the errors?
Solution 24. The errors have a pattern. The line predicts too low at both ends, below about 80 and above about 170 horsepower, and too high in the middle, because mpg falls steeply at first and then flattens.
Exercise 25 (Is a straight line the right shape? (part 2)) Use the setup of Exercise 24.
Then tick least-squares curve, which fits \(\hat{y} = \hcoef{0} + \hcoef{1} x + \hcoef{2} x^2\) by least squares. Should we always prefer the model with the lower sRMSE (Definition 22)?
Solution 25.
{
const ls = (x) => lsLine.b0 + lsLine.b1 * x;
const quad = (x) => lsQuad[0] + lsQuad[1] * x + lsQuad[2] * x ** 2;
const lsRmse = rmseOf(ls);
const quadRmse = rmseOf(quad);
const signB1 = lsQuad[1] < 0 ? "−" : "+";
const signB2 = lsQuad[2] < 0 ? "−" : "+";
const out = md`The curve
$$
\\hat{y} = ${lsQuad[0].toFixed(2)} ${signB1} ${Math.abs(lsQuad[1]).toFixed(4)}x ${signB2} ${Math.abs(lsQuad[2]).toFixed(6)}x^2
$$
follows that bend, and its sRMSE is ${quadRmse.toFixed(2)}, against ${lsRmse.toFixed(2)} for the line.`;
// Same deferred-retypeset reason as the cell in #sol-fit-a-line: MathJax
// never sees this $$...$$ unless something asks it to look again.
requestAnimationFrame(() => window.Quarto?.typesetMath?.(out));
return out;
}But a lower sRMSE on the data we fit is not the goal. A curve with enough terms could pass through every car and reach an sRMSE of zero, and still predict the next car badly. Minimizing sRMSE on the data we fit repeats the mistake of trusting the best spam rule’s score on half A. The error that matters is the error on data the rule has not seen, which is generalization, the subject of A First Model and Generalization.
Abu-Mostafa sorts learning problems into supervised, unsupervised and reinforcement learning in his first Caltech lecture, from 36:04 [8], Lecture 1.
A program that plays checkers has to choose a move on every turn. One way to choose is to look at the positions each move could lead to, score each position, and pick the move whose position scores best.
Exercise 26 (Describe a checkers position with numbers) Suppose the program plays Black. List four numbers you could compute from a checkers board that say whether the position is good for Black.
Think about what a checkers player counts or looks at before deciding who is ahead.
Solution 26. Four good choices:
The IBM checkers program used all four, from a list of 38 such features plus piece advantage [30], “Term replacement”.
Exercise 27 (What does scoring a position take?) To pick a move, the program scores each position it could reach. What goes into that scoring rule, and what comes out?
Solution 27. A board position goes in, and one number comes out. A bigger number should mean the position is better for Black. So the scoring rule is a function from positions to numbers.
Definition 23 (Evaluation function) An evaluation function \(V\) assigns a number \(V(s)\) to each game position \(s\), with a larger number meaning the position is better for the player it scores [31].
Example 15 (Scoring a position by who has more pieces) Score a position \(s\) from Black’s side with \[ V(s) = \begin{cases} 1 & \text{if Black has more pieces than White,} \\ 0 & \text{if the two sides have the same number of pieces,} \\ -1 & \text{if Black has fewer pieces than White.} \end{cases} \] A position \(s\) where Black has 7 pieces and White has 5 scores \(V(s) = 1\).
Exercise 28 (Combine the features into one score) How could the program combine the numbers from Exercise 26 into a single score for a position?
Solution 28. Give each feature a weight, multiply each feature by its weight, and add up the products. A feature that matters more for winning gets a larger weight.
Definition 24 (Linear evaluation function) A linear evaluation function is an evaluation function (Definition 23) that is a weighted sum, the dot product of a weight vector with a feature vector (Definition 8) describing the position, \[ V_{w}(s) = \sum_{j=1}^{\ninputvars} w_j x_j(s), \] where \(x_1(s), \ldots, x_{\ninputvars}(s)\) are the features of \(s\), and the numbers \(w_1, \ldots, w_{\ninputvars}\) are its weights.
Example 16 (A two-feature evaluation function) Take two features, \(x_1(s)\), Black’s piece advantage, and \(x_2(s)\), Black’s mobility advantage (the number of squares Black can move a piece to, minus White’s number), with weights \(w_1 = 3\) and \(w_2 = 1\). A position \(s\) where Black is ahead by one piece and can move to 5 more squares than White scores \[ \begin{aligned} V_{w}(s) &= 3 \times 1 + 1 \times 5 \\ &= 8. \end{aligned} \]
The features say what the program looks at. The weights say how much each feature matters, and the program can learn the weights.
Exercise 29 (Where can the weights come from?) Name three ways to choose the weights \(w_1, \ldots, w_{\ninputvars}\) of a linear evaluation function for checkers (Definition 24).
Solution 29.
Exercise 30 (Where can the weights come from? (part 2)) Use the setup of Exercise 29.
Which of your three ways count as machine learning?
Solution 30. Ways 2 and 3 are machine learning: the program’s play improves with experience, and the experience is data, not rules a person wrote down. The IBM checkers program learned its weights by way 3 in 1959 [30]. A 1967 follow-up reported an improved “book learning” procedure, which Samuel judged “much superior to the linear polynomial method” [32], abstract.
Nobody tells a program playing against itself how good a position really is, or which move was correct. Learning from self-play is a kind of reinforcement learning (Definition 16), and the program has to make its own targets.
Exercise 31 (Score a position by looking ahead) The evaluation function scores a position from its features alone. How could the program use the evaluation function to get a more reliable score for a position \(s\), by looking several moves ahead of \(s\)?
Solution 31. Look at every position the game could reach from \(s\) within a few moves, and score each of those positions with the evaluation function. Then work backwards, assuming each player picks the move that is best for that player.
Definition 25 (Backed-up score) The backed-up score of a position \(s\) looks several moves ahead: it applies the evaluation function to every position reachable from \(s\) within a fixed number of moves, then assumes each side picks the move best for itself, working back one move at a time until it reaches \(s\) [33].
Example 17 (A two-move backed-up score) In position \(s\), Black is to move and has two moves, A and B. White then has two replies to each. The evaluation function scores the four resulting positions for Black:
| Black’s move | White’s reply | score for Black |
|---|---|---|
| A | A1 | \(5\) |
| A | A2 | \(1\) |
| B | B1 | \(3\) |
| B | B2 | \(2\) |
White picks the reply that is worst for Black, so move A is worth \(\min(5, 1) = 1\) and move B is worth \(\min(3, 2) = 2\). Black picks the better of the two moves, so the backed-up score of \(s\) is \(\max(1, 2) = 2\).
Exercise 32 (A target without labels) The program has two ways to score a position \(s\): the evaluation function \(V_{w}(s)\) (Definition 24), and the backed-up score of \(s\) (Definition 25). Which of the two should the program trust more?
Solution 32. The backed-up score is more trustworthy.
Exercise 33 (A target without labels (part 2)) Use the setup of Exercise 32.
Why should the program trust that one more?
Solution 33. The backed-up score has seen more of the game: it scores positions several moves further on, where more of the outcome is already settled.
Exercise 34 (A target without labels (part 3)) Use the setup of Exercise 32.
How could the program use the one it trusts more to improve the weights \(w\)?
Solution 34. So the program can treat the backed-up score as the target, and change the weights until \(V_{w}(s)\) comes closer to it.
we are attempting to make the score, calculated for the current board position, look like that calculated for the terminal board position of the chain of moves which most probably will occur during actual play.
[30], “Polynomial modification procedure”
Exercise 35 (Score a position and find the gap) Use the two-feature evaluation function of Example 16, with a new position \(s\) and these values:
| symbol | meaning | value |
|---|---|---|
| \(x_1(s)\) | Black’s piece advantage in \(s\) | \(2\) |
| \(x_2(s)\) | Black’s mobility advantage in \(s\) | \(-1\) |
| \(w_1, w_2\) | current weights | \(1, 1\) |
| \(t\) | backed-up score of \(s\) (Definition 25) | \(3\) |
Find the score \(V_{w}(s)\) and the gap \(\delta = t - V_{w}(s)\) between the target and the score.
Solution 35. \[ \begin{aligned} V_{w}(s) &= w_1 x_1(s) + w_2 x_2(s) && \text{(definition of } V_w\text{)} \\ &= 1 \times 2 + 1 \times (-1) && \text{(substitute the values)} \\ &= 1, \\ \delta &= t - V_{w}(s) && \text{(definition of } \delta\text{)} \\ &= 3 - 1 && \text{(substitute the values)} \\ &= 2. \end{aligned} \]
The score is too low by 2.
Exercise 36 (Change the weights slightly) One rule for changing the weights is \[ w_j \leftarrow w_j + \eta \, \delta \, x_j(s), \] where the step size \(\eta\) controls how far each weight moves. Use \(\eta = 0.1\) and your \(\delta\) from Exercise 35 to find the new weights \(w_1\) and \(w_2\).
Solution 36. \[ \begin{aligned} w_1 &\leftarrow w_1 + \eta \, \delta \, x_1(s) && \text{(the update rule, } j = 1\text{)} \\ &= 1 + 0.1 \times 2 \times 2 && \text{(substitute the values)} \\ &= 1.4, \\ w_2 &\leftarrow w_2 + \eta \, \delta \, x_2(s) && \text{(the update rule, } j = 2\text{)} \\ &= 1 + 0.1 \times 2 \times (-1) && \text{(substitute the values)} \\ &= 0.8. \end{aligned} \]
The score was too low (\(\delta > 0\)), so the weight on the feature that raised the score (\(x_1 > 0\)) grew, and the weight on the feature that lowered the score (\(x_2 < 0\)) shrank.
Exercise 37 (Check the new score) Find the new score \(V_{w}(s)\) with the new weights from Exercise 36.
Solution 37. \[ \begin{aligned} V_{w}(s) &= w_1 x_1(s) + w_2 x_2(s) && \text{(definition of } V_w\text{)} \\ &= 1.4 \times 2 + 0.8 \times (-1) && \text{(substitute the new weights)} \\ &= 2.8 - 0.8 && \text{(multiply)} \\ &= 2.0. \end{aligned} \]
Exercise 38 (Check the new score (part 2)) Use the setup of Exercise 37.
Did the score move toward the target \(t = 3\)?
Solution 38. The new score, 2.0, is closer to \(t = 3\) than the old score of 1.
The IBM program followed the same principle with a different rule. It kept, for each feature, a correlation between the sign of that feature’s contribution to the score and the sign of \(\delta\), and set the weights to powers of 2 in proportion to those correlations. It also swapped the feature with the weakest correlation for one from a reserve list, about once every eight moves [30], “Polynomial modification procedure” and “Term replacement”. The rule in Exercise 36 is a later, simpler one.
Exercise 39 (Remember positions you have scored) Suppose the program looks 3 moves ahead, and it saves every position it meets in play, together with that position’s backed-up score (Definition 25). The 3-move lookahead in a later game reaches one of the saved positions. How far ahead does the saved score effectively look?
Solution 39. The saved score already looks 3 moves past the saved position, and the saved position is 3 moves past the current one, so the score in use looks \(3 + 3 = 6\) moves ahead. The memory grows with every game, and the effective lookahead grows with the memory.
The IBM program saved over 53,000 positions this way. It learned to play good openings and endgames, though its midgame play barely improved [30], “Rote-learning tests”.
The rest of this page is optional: we do it in class if there is time, and otherwise it is here for you to work through on your own.
Exercise 40 (Look at the spam data) The spam data are the Spambase set, collected at Hewlett-Packard Labs and published in the UCI Machine Learning Repository [7]. The same 4601 messages also ship as spam in R’s kernlab package [34], and Python’s ucimlrepo package downloads them from UCI. These notes keep a copy of the file, with UCI’s column names, at data/spambase.csv, so each spam exercise in this section that loads the data reads the same file.
Load it.
Exercise 41 (Look at the spam data (part 2)) Use the setup of Exercise 40.
Look at the george, you, free and hp columns.
Solution 41.
'data.frame': 4601 obs. of 4 variables:
$ word_freq_george: num 0 0 0 0 0 0 0 0 0 0 ...
$ word_freq_you : num 1.93 3.47 1.36 3.18 3.18 0 3.85 0 1.23 1.67 ...
$ word_freq_free : num 0.32 0.14 0.06 0.31 0.31 0 0.96 0 0 0 ...
$ word_freq_hp : num 0 0 0 0 0 0 0 0 0 0 ...
Most entries are 0: in any one message, most of the 57 words never appear.
Exercise 42 (Look at the spam data (part 3)) Use the setup of Exercise 40.
Count how many messages are spam.
Solution 42.
table(spam$Class)
0 1
2788 1813
There are 4601 messages: 1813 spam and 2788 email. The file codes spam as 1 and email as 0.
Exercise 43 (Read the file with R’s defaults) UCI names six of Spambase’s columns after punctuation marks: char_freq_;, char_freq_( and so on. Read data/spambase.csv into R with read.csv() and no other arguments, and print the names of columns 49 to 54.
Exercise 44 (Read the file with R’s defaults (part 2)) Use the setup of Exercise 43.
What happened to them?
Solution 44. Six names changed: ;, (, [, !, $ and # cannot appear in an R name, so each became ., and the six results were made unique by appending .1, .2 and so on.
Exercise 45 (Read the file with R’s defaults (part 3)) Use the setup of Exercise 43.
Why did that happen to them?
Solution 45. By default read.csv passes every column name through make.names(), which turns anything that is not a legal R name into one.
Exercise 46 (Read the file with R’s defaults (part 4)) Use the setup of Exercise 43.
How do you keep the names as they are in the file?
Solution 46.
[1] "char_freq_;" "char_freq_(" "char_freq_[" "char_freq_!" "char_freq_$"
[6] "char_freq_#"
check.names = FALSE keeps the file’s names, which must then be quoted with backticks in $ access: spam$`char_freq_(`.
Python’s pandas.read_csv keeps them as they are, and they are used as strings in either case: spam["char_freq_!"].
Exercise 47 (Look at four messages) The table of spam and email averages gives the average message in each group. Select the same five columns for the first two messages in the file, and for the first two real messages that contain both george and hp.
Solution 47. Python counts rows from 0 and R from 1, so the same four messages are [0, 1, 1818, 1823] in one and c(1, 2, 1819, 1824) in the other.
word_freq_george word_freq_you word_freq_free char_freq_! word_freq_hp Class
0 0.00 1.93 0.32 0.778 0.00 1
1 0.00 3.47 0.14 0.372 0.00 1
1818 2.04 0.00 0.00 0.000 4.08 0
1823 0.24 0.24 0.00 0.149 9.33 0
spam <- read.csv("data/spambase.csv", check.names = FALSE)
cols <- c("word_freq_george", "word_freq_you", "word_freq_free",
"char_freq_!", "word_freq_hp", "Class")
spam |>
dplyr::mutate(id = dplyr::row_number()) |>
dplyr::filter(id %in% c(1, 2, 1819, 1824)) |>
dplyr::select(id, dplyr::all_of(cols)) |>
knitr::kable()| id | word_freq_george | word_freq_you | word_freq_free | char_freq_! | word_freq_hp | Class |
|---|---|---|---|---|---|---|
| 1 | 0.00 | 1.93 | 0.32 | 0.778 | 0.00 | 1 |
| 2 | 0.00 | 3.47 | 0.14 | 0.372 | 0.00 | 1 |
| 1819 | 2.04 | 0.00 | 0.00 | 0.000 | 4.08 | 0 |
| 1824 | 0.24 | 0.24 | 0.00 | 0.149 | 9.33 | 0 |
Exercise 48 (Look at four messages (part 2)) Use the setup of Exercise 47.
Does each message look like its group’s average?
Solution 48.
you_spam_avg <- mean(spam$word_freq_you[spam$Class == 1])Each message does look like its group. Neither spam message contains george or hp at all, and both real ones contain both.
An average is not a guarantee, though. Message 1 is spam with you at 1.93, below the spam average of 2.26 and well below message 2’s 3.47. A rule built on one column will misclassify messages that a rule built on several would classify correctly.
Exercise 49 (Check the order of the rows) Is the file shuffled? Look at the Class column from top to bottom and say how it is arranged.
Solution 49. Count the runs of a constant label, from the top of the file down:
There are 2 runs, so the label changes exactly once: all 1813 spam messages come first, then all 2788 real ones.
Exercise 50 (Check the order of the rows (part 2)) Use the setup of Exercise 49.
Why would the order of the rows matter?
Solution 50. So taking the first 80% of the rows, 3680 of them, as a training set would put every spam message in training and leave a test set of email alone. Any split of this file has to shuffle first, so the two halves in the held-out exercise were drawn at random.
Exercise 51 (How many rules did the computer try?) The best rule search in the exercise on beating the all-email rule tried every %george threshold from 0 to 5 and every %you threshold from 0 to 10, each in steps of 0.05. Count the candidate rules.
Solution 51. From 0 to 5 in steps of 0.05 is \(5 / 0.05 + 1 = 101\) thresholds, and from 0 to 10 is \(10 / 0.05 + 1 = 201\). Every pair is a rule, so the search tried \(101 \times 201 = 20301\) rules, each scored on all 4601 emails. A computer does that in much less than a second; a person does not.
The name statistical learning is recent, but many of its methods are old [10, Ch. 1]:
Statisticians and computer scientists developed these methods at the same time. The same tools led to data mining, machine learning and bioinformatics, fields whose methods “have common underpinnings but are often expressed with different terminology” [5], back cover.
The Data Science Learning Community’s book clubs discuss An Introduction to Statistical Learning, Chapter 1, in recorded meetings:
Exercise 52 (Load the data yourself) The Auto data are available as data/auto.csv; the note on the Auto data says where that file comes from. In either language, load the file and report how many rows and columns it has.
Solution 52.
'data.frame': 392 obs. of 9 variables:
$ mpg : num 18 15 18 16 17 15 14 14 14 15 ...
$ cylinders : int 8 8 8 8 8 8 8 8 8 8 ...
$ displacement: num 307 350 318 304 302 429 454 440 455 390 ...
$ horsepower : int 130 165 150 150 140 198 220 215 225 190 ...
$ weight : int 3504 3693 3436 3433 3449 4341 4354 4312 4425 3850 ...
$ acceleration: num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ...
$ year : int 70 70 70 70 70 70 70 70 70 70 ...
$ origin : int 1 1 1 1 1 1 1 1 1 1 ...
$ name : Factor w/ 301 levels "amc ambassador brougham",..: 49 36 230 14 160 141 54 222 240 2 ...
Both find 392 cars. Python reports 8 columns, because the code makes each car’s name the row index, as ISLP does; R reports 9, with name as an ordinary column.
Exercise 53 (Summarize three variables) Now summarize mpg, horsepower and weight. What is the range of each, and the mean of mpg?
Solution 53.
mpg horsepower weight
count 392.000000 392.000000 392.000000
mean 23.445918 104.469388 2977.584184
std 7.805007 38.491160 849.402560
min 9.000000 46.000000 1613.000000
25% 17.000000 75.000000 2225.250000
50% 22.750000 93.500000 2803.500000
75% 29.000000 126.000000 3614.750000
max 46.600000 230.000000 5140.000000
Auto <- read.csv("data/auto.csv", stringsAsFactors = TRUE)
Auto |>
dplyr::select(mpg, horsepower, weight) |>
gtsummary::tbl_summary(
type = gtsummary::all_continuous() ~ "continuous2",
statistic = gtsummary::all_continuous() ~
c("{mean} ({sd})", "{min}, {max}"),
digits = gtsummary::all_continuous() ~ 1
) |>
gtsummary::as_kable()| Characteristic | N = 392 |
|---|---|
| mpg | |
| Mean (SD) | 23.4 (7.8) |
| Min, Max | 9.0, 46.6 |
| horsepower | |
| Mean (SD) | 104.5 (38.5) |
| Min, Max | 46.0, 230.0 |
| weight | |
| Mean (SD) | 2,977.6 (849.4) |
| Min, Max | 1,613.0, 5,140.0 |
mpg runs from 9 to 46.6, with a mean of 23.4; horsepower from 46 to 230; and weight from 1613 to 5140 pounds.
The chapter 2 lab goes further, in Python (Ch02-statlearn-lab.ipynb in the ISLP labs repository) and in R (the chapter 2 files on the book’s R resources page).
Exercise 54 (Supervised or unsupervised?) For each task, decide whether it is supervised or unsupervised.
Recognize which digit, 0 to 9, a handwritten image shows.
Solution 54. Supervised: each image comes with its digit.
Exercise 55 (Supervised or unsupervised? (part 2)) Use the setup of Exercise 54.
Predict tomorrow’s high temperature in Bellingham.
Solution 55. Supervised: past days come with their recorded highs.
Exercise 56 (Supervised or unsupervised? (part 3)) Use the setup of Exercise 54.
Group news articles by topic, with no topics given in advance.
Solution 56. Unsupervised (clustering): no topics are given.
Exercise 57 (Supervised or unsupervised? (part 4)) Use the setup of Exercise 54.
Predict a house’s sale price from its size and location.
Solution 57. Supervised: past sales come with their prices.
Exercise 58 (Supervised or unsupervised? (part 5)) Use the setup of Exercise 54.
Decide whether a credit-card transaction is fraudulent, given past transactions marked as fraud or not.
Solution 58. Supervised: past transactions are marked as fraud or not.
The fraud task can also be an unsupervised problem. A detector could flag transactions unlike the rest even with no labels marking past fraud. That version of the task is called anomaly detection. The same goal can lead to different problems depending on what data exist.
Exercise 59 (Supervised or unsupervised? (part 6)) Use the setup of Exercise 54.
Find groups of customers who shop in similar ways.
Solution 59. Unsupervised (clustering): no groups are given.
Exercise 60 (Regression or classification, again) For each supervised task in Exercise 54, decide whether it is regression or classification.
Solution 60.
Exercise 61 (Asking questions of data) Machine learning starts with a question that data can help answer. A transit agency records card taps at stations, with the time, the line and the weather that day. What patterns might exist in these data?
Solution 61. Many answers work. Some examples:
Public transit: the traffic at a station may depend on the time, the line and the weather, and the traffic at some stations may rise and fall together.
Exercise 62 (Asking questions of data (part 2)) What would you want a model to predict from the transit data of Exercise 61?
Solution 62. Public transit: predict how many people will tap in at a station during the next hour from the time, the line and the weather.
Exercise 63 (Asking questions of data (part 3)) Would the model of Exercise 62 be supervised or unsupervised?
Solution 63. Public transit: predicting the number of taps is supervised regression, and finding groups of stations whose traffic rises and falls together is unsupervised.
Exercise 64 (Asking questions of data (part 4)) A farm records soil samples, planting dates, rainfall and harvest weights for many fields. What patterns might exist in these data?
Solution 64. Farming: a field’s harvest weight may depend on its soil, planting date and rainfall, and fields planted late with little rain may form a group.
Exercise 65 (Asking questions of data (part 5)) What would you want a model to predict from the farming data of Exercise 64?
Solution 65. Farming: predict a field’s harvest weight from its soil, planting date and rainfall.
Exercise 66 (Asking questions of data (part 6)) Would the model of Exercise 65 be supervised or unsupervised?
Solution 66. Farming: predicting the harvest weight is supervised regression, and flagging fields that were planted late and got little rain is unsupervised grouping.
Exercise 67 (Asking questions of data (part 7)) A public library records each loan, with the book’s subject and the borrower’s earlier loans. What patterns might exist in these data?
Solution 67. A public library: borrowers who read one subject may borrow more books on it, and some borrowers may read alike.
Exercise 68 (Asking questions of data (part 8)) What would you want a model to predict from the library data of Exercise 67?
Solution 68. A public library: predict whether a borrower will take out a second book from the same subject.
Exercise 69 (Asking questions of data (part 9)) Would the model of Exercise 68 be supervised or unsupervised?
Solution 69. A public library: predicting whether a borrower takes a second book is supervised classification, and grouping borrowers by what they read is unsupervised.
Exercise 70 (Your own problem) Brainstorm one use of machine learning not mentioned in this lecture, and name its input features and its output.
Solution 70. Answers will vary; we will share a few and sort them together, as supervised or unsupervised. For a supervised one, check that its output is something you could have recorded for past examples, since the model learns from those recorded outputs.