Last modified: 2026-11-19
Every definition in the notes, listed by module and page in the order the notes present them. Each row numbers the definition within its page and across the whole list, so the last overall number counts every definition. The list is built from the notes themselves, so it cannot fall out of step with them. The glossary explains the key terms in plain words and points to where each is defined. The results are listed the same way.
| On page | Overall | Definition |
|---|---|---|
| 1 | 1 | Machine learning |
| 2 | 2 | Generalization |
| 3 | 3 | Generalizing well |
| 4 | 4 | Learning from experience |
| 5 | 5 | Artificial intelligence |
| 6 | 6 | Prediction |
| 7 | 7 | Inference |
| On page | Overall | Definition |
|---|---|---|
| 1 | 8 | Training data (training set) |
| 2 | 9 | Learning algorithm |
| 3 | 10 | Training (fitting, learning, estimation) |
| 4 | 11 | Feature (attribute) |
| 5 | 12 | Model |
| 6 | 13 | Statistical model |
| 7 | 14 | Predictive model |
| 8 | 15 | Feature vector |
| 9 | 16 | Feature function (feature map, encoding) |
| 10 | 17 | Output variable (label, target, response, outcome, dependent variable) |
| 11 | 18 | Structured data |
| 12 | 19 | Unstructured data |
| 13 | 20 | Supervised learning |
| 14 | 21 | Unsupervised learning |
| 15 | 22 | Semi-supervised learning |
| 16 | 23 | Reinforcement learning |
| 17 | 24 | Clustering |
| 18 | 25 | k-means |
| 19 | 26 | Regression |
| 20 | 27 | Classification |
| 21 | 28 | Sample mean squared error |
| 22 | 29 | Sample root mean squared error |
| 23 | 30 | Evaluation function |
| 24 | 31 | Linear evaluation function |
| 25 | 32 | Backed-up score |
| On page | Overall | Definition |
|---|---|---|
| 1 | 67 | Interaction term (product term, cross term) |
| 2 | 68 | Hierarchy principle (hierarchical principle) |
| 3 | 69 | Standard error of a coefficient |
| 4 | 70 | Estimated standard error |
| 5 | 71 | Confidence interval for a coefficient |
| 6 | 72 | The \(t\)-test for a coefficient |
| 7 | 73 | The \(F\)-test that any feature is useful |
| 8 | 74 | The partial \(F\)-test for a group of features |
| 9 | 75 | Confidence interval for the mean response |
| 10 | 76 | Prediction interval |
| 11 | 77 | Residual plot |
| 12 | 78 | Standardized residual |
| 13 | 79 | Outlier |
| 14 | 80 | Variance inflation factor |
| On page | Overall | Definition |
|---|---|---|
| 1 | 81 | Multivariate linear regression |
| On page | Overall | Definition |
|---|---|---|
| 1 | 82 | Parameter (weight, coefficient) |
| 2 | 83 | Training set |
| 3 | 84 | Fitted model |
| 4 | 85 | Parametric model |
| 5 | 86 | True mean function |
| 6 | 87 | Noise conditional on the input |
| 7 | 88 | Noise model |
| 8 | 89 | Loss function |
| 9 | 90 | Plausible loss |
| 10 | 91 | Friendly loss |
| 11 | 92 | Risk (expected loss, population risk, true risk, generalization error, expected prediction error) |
| 12 | 93 | Conditional risk (pointwise risk, risk at an input) |
| 13 | 94 | Empirical risk |
| 14 | 95 | Training error |
| 15 | 96 | Empirical risk minimization |
| 16 | 97 | Metric |
| 17 | 98 | Absolute metric |
| 18 | 99 | Relative metric |
| 19 | 100 | Accuracy |
| 20 | 101 | Error rate |
| 21 | 102 | Interpolation |
| On page | Overall | Definition |
|---|---|---|
| 1 | 103 | Objective function (objective) |
| 2 | 104 | Cost function |
| 3 | 105 | Validation set |
| 4 | 106 | Validation error |
| 5 | 107 | Test set |
| 6 | 108 | Test error |
| 7 | 109 | Generalization gap |
| 8 | 110 | Folds |
| 9 | 111 | \(k\)-fold cross-validation |
| 10 | 112 | \(k\)-fold cross-validation estimate |
| 11 | 113 | Leave-one-out cross-validation |
| On page | Overall | Definition |
|---|---|---|
| 1 | 114 | Prediction rule |
| 2 | 115 | Average fit |
| 3 | 116 | Bias of a prediction |
| 4 | 117 | Variance of a prediction |
| 5 | 118 | Underfitting |
| 6 | 119 | Overfitting |
| 7 | 120 | Residual variance (residual noise variance, irreducible error) |
| 8 | 121 | Best member of a model family |
| 9 | 122 | Deterministic noise |
| 10 | 123 | Regularization |
| 11 | 124 | Penalty weight (regularization strength) |
| On page | Overall | Definition |
|---|---|---|
| 1 | 125 | Sequence labeling |
| 2 | 126 | Sequence-to-sequence prediction (sequence-to-sequence decoding, seq2seq) |
| 3 | 127 | Ranking |
| 4 | 128 | Structured prediction |
| 5 | 129 | Density estimation |
| 6 | 130 | Anomaly detection (novelty detection, outlier detection) |
| 7 | 131 | Non-parametric model |
| 8 | 132 | Probabilistic model |
| 9 | 133 | Non-probabilistic model |
| 10 | 134 | Discriminative model |
| 11 | 135 | Generative model |
| 12 | 136 | One-hot encoding (indicator coding) |
| 13 | 137 | Standardization (z-scoring, standard scaling) |
| 14 | 138 | Min–max normalization (min–max scaling) |
| On page | Overall | Definition |
|---|---|---|
| 1 | 139 | Decision region |
| 2 | 140 | Decision boundary |
| On page | Overall | Definition |
|---|---|---|
| 1 | 141 | Logistic regression |
| 2 | 142 | Link function |
| 3 | 143 | Probit regression |
| 4 | 144 | Complementary log-log regression |
| On page | Overall | Definition |
|---|---|---|
| 1 | 145 | Learning rate (step size) |
| 2 | 146 | Stochastic gradient descent |
| 3 | 147 | Robbins–Monro conditions |
| 4 | 148 | Minibatch gradient descent |
| 5 | 149 | Epoch |
| 6 | 150 | Online training (online learning) |
| 7 | 151 | Patience |
| 8 | 152 | Performance decay |
| On page | Overall | Definition |
|---|---|---|
| 1 | 153 | Log loss |
| 2 | 154 | Cross-entropy loss |
| 3 | 155 | Total log loss |
| 4 | 156 | Margin of an observation |
| 5 | 157 | Cross-entropy of a classifier |
| 6 | 158 | Residual of a logistic regression |
| On page | Overall | Definition |
|---|---|---|
| 1 | 159 | Separated data (completely separated data) |
| 2 | 160 | Quasi-completely separated data |
| 3 | 161 | Hauck–Donner effect |
| On page | Overall | Definition |
|---|---|---|
| 1 | 162 | Softmax function |
| 2 | 163 | Multinomial logistic regression (softmax regression) |
| 3 | 164 | Ordinal logistic regression (cumulative logit model, proportional odds model) |
| On page | Overall | Definition |
|---|---|---|
| 1 | 165 | Bayes classifier |
| 2 | 166 | Generative classifier (plug-in Bayes classifier) |
| 3 | 167 | Linear discriminant function |
| 4 | 168 | Linear discriminant analysis (LDA) |
| 5 | 169 | Quadric surface, quadric |
| 6 | 170 | Quadratic discriminant function |
| 7 | 171 | Quadratic discriminant analysis (QDA) |
| 8 | 172 | The naive Bayes assumption |
| 9 | 173 | Naive Bayes classifier |
| 10 | 174 | Gaussian naive Bayes classifier |
| 11 | 175 | K-nearest neighbors classifier |
| On page | Overall | Definition |
|---|---|---|
| 1 | 176 | Confusion matrix for \(C\) classes |
| 2 | 177 | Confusion matrix for a detection task |
| 3 | 178 | Precision (also called positive predictive value) |
| 4 | 179 | Recall (also called sensitivity or true positive rate) |
| 5 | 180 | Specificity (also called true negative rate) |
| 6 | 181 | One-vs-rest reduction |
| 7 | 182 | Macro-averaging |
| 8 | 183 | Micro-averaging |
| 9 | 184 | Balanced accuracy |
| 10 | 185 | F-measure (also called F-score) |
| 11 | 186 | F1 score |
| 12 | 187 | Threshold |
| 13 | 188 | Precision-recall curve |
| 14 | 189 | Area under the precision-recall curve (PR AUC) |
| 15 | 190 | False positive rate (also called false alarm rate) |
| 16 | 191 | Receiver operating characteristic (ROC) curve |
| 17 | 192 | Area under the ROC curve (ROC AUC) |
| On page | Overall | Definition |
|---|---|---|
| 1 | 193 | Forward selection |
| 2 | 194 | Backward selection |
| 3 | 195 | Best subset selection |
| On page | Overall | Definition |
|---|---|---|
| 1 | 196 | Ridge regression, Tikhonov regularization, \(\lpnorm{2}\) regularization |
| 2 | 197 | Weight decay |
| 3 | 198 | Lasso (least absolute shrinkage and selection operator), \(\lpnorm{1}\) regularization |
| 4 | 199 | Soft-thresholding operator |
| 5 | 200 | Cyclic coordinate descent |
| 6 | 201 | Derived feature |
| 7 | 202 | Dimension reduction regression |
| 8 | 203 | Principal components regression (PCR) |
| 9 | 204 | Partial least squares regression (PLS) |
| 10 | 205 | High-dimensional data set |
| 11 | 206 | Elastic net |
| 12 | 207 | Data augmentation |
| On page | Overall | Definition |
|---|---|---|
| 1 | 208 | Hyperparameter, tuning parameter |
| 2 | 209 | Nested prediction function families, nested models |
| 3 | 210 | Variance of the cross-validation estimate |
| 4 | 211 | Selection bias, winner’s curse |
| On page | Overall | Definition |
|---|---|---|
| 1 | 212 | Linear classifier |
| 2 | 213 | Linearly separable |
| 3 | 214 | Separating hyperplane |
| 4 | 215 | Margin |
| 5 | 216 | Support vector |
| 6 | 217 | Max-margin hyperplane |
| 7 | 218 | Hard-margin primal problem |
| 8 | 219 | Hard-margin dual problem |
| 9 | 220 | Dual variable |
| 10 | 221 | Slack variable |
| 11 | 222 | C-SVM primal problem |
| 12 | 223 | C-SVM dual problem |
| 13 | 224 | Hinge loss |
| 14 | 225 | Kernel function |
| 15 | 226 | Kernel trick |
| 16 | 227 | One-versus-rest |
| 17 | 228 | One-versus-one |
| On page | Overall | Definition |
|---|---|---|
| 1 | 229 | Autoencoder |
| 2 | 230 | Depth |
| 3 | 231 | Width |
| 4 | 232 | Fully connected layer |
| 5 | 233 | Over-parametrized model (over-parameterized model) |
| 6 | 234 | Double descent |
| On page | Overall | Definition |
|---|---|---|
| 1 | 235 | Dynamic programming |
| 2 | 236 | Exploding gradient |
| 3 | 237 | He initialization |
| 4 | 238 | Glorot initialization |
| 5 | 239 | Gradient clipping |
| 6 | 240 | Saturated unit |
| 7 | 241 | Leaky ReLU (leaky rectifier) |
| 8 | 242 | Residual connection (skip connection, shortcut connection) |
| 9 | 243 | Saliency map |
| On page | Overall | Definition |
|---|---|---|
| 1 | 244 | Inner loop |
| 2 | 245 | Outer loop |
| 3 | 246 | Data snooping |
| 4 | 247 | Sampling bias |
| On page | Overall | Definition |
|---|---|---|
| 1 | 248 | Proportion of variance explained (explained variance ratio) |
| 2 | 249 | Cumulative proportion of variance explained |
| 3 | 250 | Scree plot |
| On page | Overall | Definition |
|---|---|---|
| 1 | 251 | Inductive bias |
| 2 | 252 | Shattering |
| 3 | 253 | Growth function |
| 4 | 254 | VC dimension |
| On page | Overall | Definition |
|---|---|---|
| 1 | 255 | K nearest neighbors |
| 2 | 256 | K-nearest neighbors regression |
| 3 | 257 | Local linear regression |
| 4 | 258 | Span |
| 5 | 259 | LOESS |
| On page | Overall | Definition |
|---|---|---|
| 1 | 260 | Step function regression, piecewise-constant regression |
| 2 | 261 | Cubic regression spline |
| 3 | 262 | Natural cubic spline |
| 4 | 263 | Smoothing spline |
| 5 | 264 | Generalized cross-validation and restricted maximum likelihood |
| On page | Overall | Definition |
|---|---|---|
| 1 | 265 | Component function |
| 2 | 266 | Additive model |
| 3 | 267 | Generalized additive model |
| 4 | 268 | Backfitting |
| On page | Overall | Definition |
|---|---|---|
| 1 | 269 | Recursive binary splitting |
| 2 | 270 | Cost-complexity pruning, weakest-link pruning |
| On page | Overall | Definition |
|---|---|---|
| 1 | 271 | Gaussian process |
| 2 | 272 | Mean function |
| 3 | 273 | Covariance function (kernel) |
| 4 | 274 | Hyperparameters |
| 5 | 275 | Marginal likelihood |
| On page | Overall | Definition |
|---|---|---|
| 1 | 276 | Flattening (vectorizing, unrolling) |
| 2 | 277 | Filter (kernel) |
| 3 | 278 | Convolution |
| 4 | 279 | Feature map (activation map, response map) |
| 5 | 280 | Convolutional layer |
| 6 | 281 | Stride |
| 7 | 282 | Padding |
| 8 | 283 | Parameter sharing (weight sharing, weight tying) |
| 9 | 284 | Sparse connectivity |
| 10 | 285 | Locally connected layer |
| 11 | 286 | Pooling (subsampling) |
| 12 | 287 | Channel |
| 13 | 288 | Recurrent layer |
| 14 | 289 | Hidden state (memory) |
| 15 | 290 | Recurrent neural network (RNN) |
| 16 | 291 | Unrolling (unfolding through time) |
| 17 | 292 | Backpropagation through time |
| 18 | 293 | Gate |
| 19 | 294 | Long short-term memory (LSTM) |
| On page | Overall | Definition |
|---|---|---|
| 1 | 295 | Perplexity |
| 2 | 296 | Substitution |
| 3 | 297 | Insertion |
| 4 | 298 | Deletion |
| 5 | 299 | Minimum edit distance alignment |
| 6 | 300 | Edit distance (Levenshtein distance) |
| 7 | 301 | Token error rate |