Notation Across Sources
This page sets our notation beside the notation of five widely used textbooks and two sets of lecture notes. The Notation Choices page gives the reasons for our conventions; this page lists the symbols side by side. The data are in data/notation-conventions.csv and data/symbols-used.csv. Links in the tables go to the definitions in the DATA 571 notes, with a few to the statistics and math prerequisite notes.
Sources
The tables on this page set our notation beside the notation of these sources:
- An Introduction to Statistical Learning, or ISL, in its R edition [1] and its Python edition [2]. They share one column because the two books use the same notation: their notation sections are word for word the same, and so are the symbols for the slack variable, the budget \(C\) and the learning rate.
- The Elements of Statistical Learning, or ESL [3].
- Learning From Data, or LfD [4].
- Bishop’s Pattern Recognition and Machine Learning [5].
- Murphy’s Probabilistic Machine Learning: An Introduction [6].
- Two sets of lecture notes from the DATA 471/571 courses, in the last two columns: Brian Hutchinson’s lecture notes [7] and Kameron Decker Harris’ 2022 weekly notes [8].
To read the tables:
- A dash means the source has no symbol for the concept or does not cover it. The printed chapters of LfD, for example, stop before neural networks and SVMs.
- A cell is bold where the source writes the symbol in bold, and plain where the source does not. Hutchinson’s notes write vectors as plain letters, and Harris’ notes put an arrow over them.
- The name of each source in a table header links to its entry in the reference list.
- A row label links to the definition in these notes where there is one.
Differences that cause confusion
Six differences cause most of the confusion when you read across sources:
- Labels: the two-class label is \(\{0, 1\}\) for logistic regression and \(\{-1, +1\}\) for SVMs in most sources, and always \(\{-1, +1\}\) in LfD.
- The SVM constant \(C\): Bishop, Murphy and Hutchinson use \(C\) as a penalty on the slack, so a larger \(C\) allows fewer violations. ISL uses \(C\) as a budget on the slack, so a larger \(C\) allows more.
- Sample index: a subscript \(i\) in ISL and ESL, a subscript \(n\) in LfD, Bishop and Murphy, and a superscript in parentheses (\(x^{(i)}\)) in Hutchinson’s notes.
- Prediction of the fitted model, averaged over data sets: these notes write the prediction at \(x\) as \(\predvalf{x}\), as ISL and ESL do (\(\hat{f}(x)\)). Where \(x\) is fixed and its dependence is not the point, as in the bias–variance decomposition, they drop the argument and write \(\predval\). Either way the prediction is a random variable, because it depends on the training set. The average fit \(\avgfitf{x}\) is the population expectation of that prediction over training sets, written with no bar and no hat.
- Target: Bishop calls the output \(t\) for target, and every other source calls it \(y\).
- Subscripts and parentheses: which arguments go in a subscript and which in parentheses varies, as in \(f_\theta(x)\) versus \(f(x; \theta)\). The Notation Choices page states our convention and the alternatives.
Tables
for (g in names(notation_groups)) {
cat("\n\n## ", notation_groups[[g]], " {#sec-notation-", g, "}\n\n", sep = "")
cat(notation_table(g), "\n", sep = "\n")
}Data
| Concept | These notes | ISL (R, Python) | ESL | LfD | Bishop | Murphy | Hutchinson | Harris |
|---|---|---|---|---|---|---|---|---|
| Number of observations | \(\nobs\) (\(N\) in the SVM chapter) | \(n\) | \(N\) | \(N\) | \(N\) | \(N\) | \(N\) | \(n\) |
| Number of input variables | \(\inputdim\) | \(p\) | \(p\) | \(d\) | \(D\) | \(D\) | \(D\) | \(d\) |
| Vector notation | a tilde under the letter: \(\vx\) | plain letter for a feature vector: \(x\); bold for a vector of length \(n\): \(\mathbf{a}\) | plain letter: \(x_i\); bold for a vector of length \(N\): \(\mathbf{x}_j\) | bold: \(\mathbf{x}\) | bold lowercase: \(\mathbf{x}\) | bold lowercase: \(\mathbf{x}\) | plain letter: \(x\) | an arrow: \(\overrightarrow{x}\) |
| Input (feature vector), observed | \(\vpredvar \in \reals^\inputdim\) | \(x_i\) | \(x_i\) | \(\mathbf{x}\), with \(x_0 = 1\) | \(\mathbf{x}\) | \(\mathbf{x}\) | \(x \in \reals^D\) | \(\overrightarrow{x}_i \in \mathbb{R}^d\) |
| Input (feature vector), random | \(\vPredvar\) | \(X\) | \(X\) | no separate symbol | no separate symbol | no separate symbol | no separate symbol | no separate symbol |
| Data matrix | \(\mX\) (\(\nobs \times \inputdim\)); \(\design\) (\(\nobs \times \nparams\)) after a feature map | \(\mathbf{X}\) (\(n \times p\)) | \(\mathbf{X}\) (\(N \times p\)) | \(X\) (\(N \times (d+1)\)) | \(\mathbf{X}\) (\(N \times D\)) | \(\mathbf{X}\) (\(N \times D\)) | \(X \in \reals^{N \times (D+1)}\), rows padded with a 1 | \(X\) (\(n \times (d+1)\)), rows padded with a 1 |
| Feature map | \(\vfeatmap\) | basis functions \(b_j\) | basis functions \(h_m\) | \(\Phi\) | \(\boldsymbol{\phi}\) (design matrix \(\boldsymbol{\Phi}\)) | \(\boldsymbol{\phi}\) | \(\phi\) | \(\Phi\) (feature matrix) |
| Output (response, target), observed | \(\outvar\) | \(y_i\) | \(y_i\) | \(y\) | \(t\) (target) | \(y\) | \(y\) | \(y_i\) |
| Output (response, target), random | \(\Outvar\) | \(Y\) | \(Y\) | no separate symbol | no separate symbol | no separate symbol | no separate symbol | \(Y\) |
| \(i\)th example | \(\vpredvar_i\), \(\outvar_i\) (\(\vpredvar_{(i)}\), \(\outvar_{(i)}\) in the SVM chapter) | \(x_i\), \(y_i\) | \(x_i\), \(y_i\) | \(\mathbf{x}_n\), \(y_n\) | \(\mathbf{x}_n\), \(t_n\) | \(\mathbf{x}_n\), \(y_n\) | \(x^{(i)}\), \(y^{(i)}\) | \(\overrightarrow{x}_i\), \(y_i\) |
| Training data | \(\trainset\) | — | \(\mathcal{T}\) | \(\mathcal{D}\) | \(\mathbf{X}\), \(\mathbf{t}\) | \(\mathcal{D}\) | \(\mathcal{X}\) | no separate symbol |
| Validation data | \(\valset\) | validation set | validation set | \(\mathcal{D}_{\mathrm{val}}\) | validation set | validation set | dev set | train blocks (\(K\)-fold) |
| Test data | \(\testset\) | test observations | test set | test set | test set | \(\mathcal{D}_{\mathrm{test}}\) | test set | test block (\(K\)-fold) |
Models, loss and risk
| Concept | These notes | ISL (R, Python) | ESL | LfD | Bishop | Murphy | Hutchinson | Harris |
|---|---|---|---|---|---|---|---|---|
| Model | \(\modmem\) (family \(\modelfam\)) | \(f\) | \(f\) | \(h \in \mathcal{H}\) (target \(f\)) | \(y(\mathbf{x}, \mathbf{w})\) | \(f(\mathbf{x}; \boldsymbol{\theta})\) | \(h\) | \(f\) |
| Fitted model | \(\emodmem\) | \(\hat{f}\) | \(\hat{f}\) | \(g\) | \(y(\mathbf{x}, \mathbf{w}^{\star})\) | \(\hat{f}\) | \(h\) | \(f\) (the fit model) |
| Fitted value | \(\eoutvar_i = \emodmemf{\predvar_i}\) | \(\hat{y}_i\) | \(\hat{y}_i\) | \(g(\mathbf{x}_n)\) | \(y(\mathbf{x}_n, \mathbf{w}^{\star})\) | \(\hat{y}_n\) | \(h(x^{(i)})\) | \(\hat{y}\) |
| Parameters, general | \(\param\) | \(\theta\) | \(\theta\) | \(\mathbf{w}\) | \(\mathbf{w}\) | \(\boldsymbol{\theta}\) | \(\theta\) | no separate symbol |
| Noise | \(\noise\) | \(\epsilon\) | \(\varepsilon\) | noisy target \(P(y \mid \mathbf{x})\) | \(\epsilon\) | \(\epsilon\) | \(\epsilon\) | \(\varepsilon\) |
| Loss on one example | \(\lossf{\outvar, \modmemf{\vx}}\) | squared error, no general symbol | \(L(Y, f(X))\) | \(e(h(\mathbf{x}), f(\mathbf{x}))\) | \(L(t, y(\mathbf{x}))\) | \(\ell(y, \hat{y})\) | \(L(y, h(x))\) | \(L(y, f(\overrightarrow{x}))\); cost \(C(\overrightarrow{\beta})\) summed over examples |
| Population (true) risk | \(\poprisk\) | expected test MSE | \(\mathrm{Err}\), \(\mathrm{EPE}\) | \(E_{\mathrm{out}}\) | \(\mathbb{E}[L]\) | \(L(\boldsymbol{\theta}; p^{*})\) | \(R(h)\) | \(E(\hat{y} - y)^2\), no name |
| Training error (empirical risk on the training set) | \(\empriskf{\modmem; \trainset}\) | training MSE | \(\overline{\mathrm{err}}\) | \(E_{\mathrm{in}}\) | \(E(\mathbf{w})\) | \(L(\boldsymbol{\theta}; \mathcal{D}_{\mathrm{train}})\) | \(\hat{R}(h)\) | \(\mathrm{MSE}\), cost \(C\) |
| Regularization strength | \(\regpar\) | \(\lambda\) | \(\lambda\) | \(\lambda\) | \(\lambda\) | \(\lambda\) | \(\lambda\) | \(\lambda\) |
| Learning rate (step size) | \(\learnrate\) | \(\rho\) | \(\gamma_r\) | \(\eta\) | \(\eta\) | \(\eta\) | \(t_k\) (step size) | \(h\) |
Linear models
| Concept | These notes | ISL (R, Python) | ESL | LfD | Bishop | Murphy | Hutchinson | Harris |
|---|---|---|---|---|---|---|---|---|
| Regression coefficients | \(\vcoef\) | \(\beta\) | \(\beta\) | \(\mathbf{w}\) | \(\mathbf{w}\) | \(\mathbf{w}\) | \(\beta\) | \(\overrightarrow{\beta}\) |
| Weights (neural networks, SVMs) | \(\vw\) | \(w_{kj}\), \(\beta_k\) (neural network), \(\beta_j\) (SVM) | \(\alpha_m\), \(\beta_k\) (neural network), \(\beta\) (SVM) | \(\mathbf{w}\) | \(\mathbf{w}\) | \(\mathbf{w}\) | \(w\) (SVM), \(W\), \(U\) (neural network) | \(w_i\) (hidden units), \(\overrightarrow{\beta}\) (output layer) |
| Intercept or bias | \(\intcoef\) (regression), \(b\) (neural networks, SVMs) | \(\beta_0\) | \(\beta_0\) | \(w_0\), with \(x_0 = 1\) | \(w_0\) (\(b\) in the SVM chapter) | \(b\) (\(w_0\) in the SVM chapter) | \(\beta_0\) or \(b\), or absorbed by padding \(\hat{x} = [1; x]\) | \(\beta_0\) |
| Logistic (sigmoid) function | \(\sigmoid\) | no symbol | no symbol | \(\theta(s)\) | \(\sigma(a)\) | \(\sigma(a)\) | \(\sigma\) | no symbol (the formula is written out) |
| Odds | \(\odds\) | no symbol (the ratio \(p(X)/[1-p(X)]\) is written out) | no symbol | — | no symbol | no symbol | — | no symbol |
| Log-odds | \(\logodds\) | no symbol (“log odds” in words) | no symbol (the logit) | — | \(a\) | \(a\) | — | \(f(\vec{x})\) |
| Odds ratio | \(\ror\) | no symbol (“log odds ratio” in words) | no symbol | — | no symbol | no symbol | — | no symbol |
| Class labels, two classes | \(\{0, 1\}\) (logistic regression), \(\{-1, +1\}\) (SVMs) | \(\{0, 1\}\) (logistic regression), \(\{-1, +1\}\) (SVMs) | \(\{0, 1\}\) (logistic regression), \(\{-1, +1\}\) (SVMs) | \(\{-1, +1\}\) | \(\{0, 1\}\) (logistic regression), \(\{-1, +1\}\) (SVMs) | \(\{0, 1\}\) or \(\{-1, +1\}\) | \(\{0, 1\}\) (logistic regression), \(\{-1, +1\}\) (SVMs) | \(\{+1, -1\}\) (logistic regression) |
Neural networks
| Concept | These notes | ISL (R, Python) | ESL | LfD | Bishop | Murphy | Hutchinson | Harris |
|---|---|---|---|---|---|---|---|---|
| Hidden-layer values | \(\vh\) | \(A_k\) (activations) | \(Z_m\) | — | \(z_j\) | \(\mathbf{z}_\ell\) | \(h\) | \(\phi_i\), \(\overrightarrow{\phi}\) |
| Backpropagation error signal | \(\backerr\) | — | \(\delta_{ki}\), \(s_{mi}\) | — | \(\delta_j\) | Jacobian products, no symbol | — | — |
Support vector machines
| Concept | These notes | ISL (R, Python) | ESL | LfD | Bishop | Murphy | Hutchinson | Harris |
|---|---|---|---|---|---|---|---|---|
| Margin slack | \(\slack\) | \(\epsilon_i\) | \(\xi_i\) | — | \(\xi_n\) | \(\xi_n\) | \(\xi\) | — |
| Dual variable (Lagrange multiplier) | \(\dualvar\) | \(\alpha_i\) | \(\alpha_i\) | — | \(a_n\) | \(\alpha_n\) | \(\alpha_i\) | — |
| Kernel | \(K(x, z)\) | \(K(x_i, x_{i'})\) | \(K(x, x')\) | — | \(k(\mathbf{x}, \mathbf{x}')\) | \(K(\mathbf{x}, \mathbf{x}')\) | \(K(x, z)\) | \(K(x, x')\) |
Unsupervised learning
| Concept | These notes | ISL (R, Python) | ESL | LfD | Bishop | Murphy | Hutchinson | Harris |
|---|---|---|---|---|---|---|---|---|
| Number of clusters or components | \(K\) | \(K\) | \(K\) | — | \(K\) | \(K\) | \(K\) | \(K\) |
| Mean and covariance matrix | \(\mean\), \(\covmat\) | \(\mu\), \(\Sigma\) | \(\mu\), \(\Sigma\) | — | \(\boldsymbol{\mu}\), \(\boldsymbol{\Sigma}\) | \(\boldsymbol{\mu}\), \(\boldsymbol{\Sigma}\) | \(\mu\), \(\Sigma\) | \(\mu\), \(\sigma^2\) |
| Responsibility of component \(c\) for point \(i\) | \(\resp_{ic}\) | — | \(\hat{\gamma}_i\) | — | \(\gamma(z_{nk})\) | \(r_{nk}\) | \(\gamma_{ic}\) | — |
Other comparisons
Subscripts and parentheses
The Notation Choices page gives the reason for our split.
| object | our form | other forms you may encounter |
|---|---|---|
| entry of a vector or matrix | \(x_i\), \(A_{ij}\) | \(x(i)\), \(A(i, j)\), and A[i, j] in code |
| term of a sequence | \(a_n\) | \(a(n)\) |
| function with a parameter | \(f_\theta(x)\) | \(f(x; \theta)\), \(f(x \mid \theta)\), \(f(x, \theta)\) |
| probability under a parameter | \(\Pr_\theta(A)\) | \(\Pr(A; \theta)\), \(\Pr(A \mid \theta)\) |
| derivative of order \(k\) | \(f^{(k)}(x)\) | \(f^{[k]}(x)\), \(\mathrm{D}^k f(x)\) |
| order statistic | \(X_{(i)}\) | \(X_{i:n}\) |
| sample index | \(x_i\) | \(x^{(i)}\) |
Information matrices
Sources disagree on the symbols for the observed and expected information (Table 8).
Other symbols used
The side-by-side tables show the symbols that differ between sources. This table lists every other symbol that a chapter of these notes uses, with what it means. Where a symbol takes arguments, the letters \(a\), \(b\) and so on stand in for them.
symbols_used <- utils::read.csv(
"data/symbols-used.csv",
stringsAsFactors = FALSE,
encoding = "UTF-8",
na.strings = character(0)
)
symbols_used$meaning[is.na(symbols_used$meaning)] <- ""
symbols_table <- data.frame(
Symbol = sprintf("$%s$", symbols_used$symbol),
Meaning = gsub("|", "\\|", symbols_used$meaning, fixed = TRUE)
)
knitr::kable(
symbols_table,
format = "pipe",
caption = "Other symbols used in these notes {#tbl-symbols-used}"
)| Symbol | Meaning |
|---|---|
| \(\truthind\) | a 0/1 truth indicator written delta(condition) |
| \(\mbackerr\) | a batch matrix of backpropagation error signals |
| \(\thresh\) | a classification threshold |
| \(\condnum\) | a condition number |
| \(\corpar\) | a correlation parameter |
| \(\Eigvals\) | a diagonal matrix of eigenvalues |
| \(\clipthresh\) | a gradient-clipping threshold |
| \(\bandwidth\) | a kernel bandwidth |
| \(\lagmult\) | a Lagrange multiplier |
| \(\decay\) | a multiplicative learning-rate decay factor |
| \(\unitthresh\) | a neuron’s firing threshold |
| \(\singval\) | a singular value |
| \(\smallconst\) | a small positive constant (a small step, cutoff or half-width) |
| \(\sdpar\) | a standard-deviation parameter |
| \(\varpar\) | a variance parameter |
| \(\vbackerr\) | a vector of backpropagation error signals |
| \(\vfeatmap\) | a vector of feature-map outputs (hidden-unit features) |
| \(\vnoise\) | a vector of model error terms |
| \(\vprob\) | a vector of probabilities |
| \(\abs{a}\) | absolute value |·| |
| \(\votewt\) | an AdaBoost vote weight |
| \(\wterr\) | an AdaBoost weighted training error |
| \(\angpar\) | an angle |
| \(\attnweight\) | an attention weight |
| \(\eigval\) | an eigenvalue |
| \(\hcovmat\) | an estimated covariance matrix |
| \(\hmean\) | an estimated mean |
| \(\hparam\) | an estimated parameter |
| \(\hsdpar\) | an estimated standard deviation |
| \(\tolin\) | an input radius (the delta of an epsilon-delta statement), or a small allowed failure probability |
| \(\tolout\) | an output tolerance (the epsilon of an epsilon-delta statement) |
| \(\argmax\) | arg max operator |
| \(\argmin\) | arg min operator |
| \(\paren{a}\) | auto-sized parentheses around argument |
| \(\sb{a}\) | auto-sized square brackets around argument |
| \(\avgfitf{a}\) | average fit at input x: the population expectation over training sets of the fitted model’s value at x, mu_h(x) |
| \(\avgfit\) | average fit: the population mean over training sets of the fitted model, written as the population mean with the model as subscript (mu_h; no bar or hat) |
| \(\avgiN\) | average over i from 1 to N: one over N times the sum |
| \(\avgin\) | average over i from 1 to n: one over n times the sum |
| \(\bestmodf{a}\) | best-in-family model applied to its argument, h*(·) |
| \(\bestmod\) | best-in-family model h*: the member of the mean-function model family closest to the target function |
| \(\vbeta\) | beta coefficient vector |
| \(\bias{a}\) | bias function |
| \(\Covs{a}{b}\) | bivariate covariance Cov(·,·) |
| \(\vc{a}\) | bold vector notation for argument |
| \(\clusterassignf{a}\) | cluster assigned to the argument, C applied to it |
| \(\clusterassign\) | cluster-assignment function (the encoder) of a clustering method such as k-means, C |
| \(\condE{a}{b}\) | conditional expected value E[A | B] |
| \(\Expfc{a}{b}\) | conditional expected value E_c[·] |
| \(\Cov{a}\) | covariance function Cov(·) |
| \(\dataset\) | data set: the whole collection of labelled observations that the training, validation and test sets are subsets of |
| \(\popdevf{a}\) | deviation of the argument from its population mean, ν_z |
| \(\diag\) | diagonal matrix operator (diag) |
| \(\diagf{a}\) | diagonal of matrix applied to argument |
| \(\dist\) | distributes as relation (∼) |
| \(\dprod{a}{b}\) | dot product of two arguments a·b |
| \(\emprisk\) | empirical risk R̂ |
| \(\eqdef\) | equals-by-definition symbol (≝) |
| \(\erf{a}\) | error function ε(·); for an estimate, the estimation error (estimate minus truth) |
| \(\err\) | error term ε |
| \(\veps\) | error vector (ε-vector) |
| \(\est{a}\) | estimated / hat version of argument |
| \(\hb\) | estimated beta coefficient β̂ |
| \(\hvb\) | estimated beta vector (β̂-vector) |
| \(\hvcoef\) | estimated coefficient vector |
| \(\covh{a}\) | estimated covariance function Ĉov(·) |
| \(\epredfn\) | estimated prediction function, such as one fitted from training data |
| \(\hcoef{a}\) | estimated regression coefficient for covariate (β̂_x) |
| \(\hintcoef\) | estimated regression intercept (β̂_0) |
| \(\hse{a}\) | estimated standard error ŝe(·) |
| \(\hVar{a}\) | estimated variance function V̂ar(·) |
| \(\norm{a}\) | Euclidean norm ‖·‖ |
| \(\expt\) | exp operator symbol |
| \(\Ep\) | expectation operator |
| \(\Epredvalf{a}\) | expected predicted value at the covariate value given as the argument: the population expectation over training sets of the fitted model’s prediction at that value |
| \(\Epredval\) | expected predicted value when no covariate value is salient |
| \(\Expf{a}\) | expected value |
| \(\E{a}\) | expected value E[·] |
| \(\expf{a}\) | exponential function |
| \(\vd\) | exposure vector d lowercase (d̰) |
| \(\failprob\) | failure probability delta of a probabilistic bound (one minus the confidence level) |
| \(\hy\) | fitted / predicted value ŷ |
| \(\fitmodf{a}\) | fitted model obtained from the training set given as the argument, the prediction function ŷ with the training set as subscript |
| \(\hf\) | fitted prediction function (f̂) |
| \(\hfx{a}\) | fitted prediction function applied to its argument, f̂(·) |
| \(\floor{a}\) | floor function ⌊·⌋ |
| \(\hR\) | Gelman-Rubin convergence diagnostic (potential scale reduction factor, R-hat) |
| \(\grad\) | gradient operator ∇ |
| \(\gg\) | Greek letter gamma |
| \(\growthfnf{a}\) | growth function at n points, m_Y(n) |
| \(\hatmat\) | hat matrix H, which maps observed responses to fitted responses |
| \(\h\) | hazard function symbol h |
| \(\cutheight\) | height h at which a dendrogram is cut |
| \(\imgheight\) | height of an image or feature map, in pixels |
| \(\vxi\) | i-th observation vector x̰_i |
| \(\siid\) | i.i.d. distributed relation |
| \(\testidx\) | index set of the test set |
| \(\trainidx\) | index set of the training set |
| \(\validx\) | index set of the validation set |
| \(\indicp{a}\) | indicator function 𝟙(·) with parentheses |
| \(\indic{a}\) | indicator function 𝟙_{·} (blackboard bold) |
| \(\mpredvar\) | input matrix: the predictor vectors of all n observations, one per row |
| \(\levelsetof{a}{b}\) | level set of the function given as the second argument at the value given as the first, the preimage of that single value |
| \(\preimage{a}{b}\) | preimage of the set given as the second argument under the function given as the first |
| \(\leverage{a}\) | leverage of the observation given as the argument, h with the observation as subscript |
| \(\llik\) | log-likelihood symbol (ℓ) |
| \(\logodds\) | log-odds symbol η |
| \(\sigmoidf{a}\) | logistic (sigmoid) function applied to its argument, expit(·) |
| \(\logit\) | logit function |
| \(\loss\) | loss function symbol (L) |
| \(\obsloss{a}\) | loss on one observation, ℓ_i (argument: the observation index) |
| \(\machineps\) | machine epsilon |
| \(\cent{a}\) | mark for the centered version of the argument (the data with its mean subtracted) |
| \(\matr{a}\) | matrix notation (bold style) |
| \(\trf{a}\) | matrix trace applied to argument |
| \(\tr\) | matrix trace operator (tr) |
| \(\avgleverage\) | mean leverage, h-bar |
| \(\vm\) | mean vector (μ-vector) |
| \(\modmemof{a}\) | member (or fit) h of the mean-function model family given as the argument, h with the family as subscript |
| \(\inv{a}\) | multiplicative inverse (·)^{−1} with parentheses |
| \(\lpnorm{a}\) | name of an ℓ_p norm or penalty, such as ℓ_1 or ℓ_2 (argument: p) |
| \(\ndist{a}\) | Normal distribution expression N(·) |
| \(\Normal\) | Normal distribution label N |
| \(\nmodels\) | number M of models (classifiers) in a model family |
| \(\odds\) | odds symbol ω |
| \(\vy\) | outcome vector y (y̰) |
| \(\pt{a}\) | parenthesized transpose (x)′ |
| \(\deriv{a}\) | partial derivative operator ∂/∂(·) |
| \(\derivf{a}{b}\) | partial derivative ∂f/∂x with numerator |
| \(\popdist\) | population distribution P that the data are drawn from |
| \(\pMSE{a}\) | population mean squared error pMSE(·) applied to argument |
| \(\popriskf{a}\) | population risk of a model, R(·) |
| \(\Prf{a}\) | Pr(·) probability expression |
| \(\predval\) | predicted value of the outcome |
| \(\predvalf{a}\) | prediction at the covariate value given as the argument, from the model that a fitting procedure fits to the training set |
| \(\prederr\) | prediction error, predicted minus observed outcome |
| \(\predfnf{a}\) | prediction function applied to the argument |
| \(\predfam\) | prediction function family Y-script: the set of prediction functions that training chooses from |
| \(\Predvar\) | predictor variable, random variable |
| \(\Pf{a}\) | probability P(·) |
| \(\prob\) | probability symbol π |
| \(\prodin\) | product over i from 1 to n |
| \(\coef{a}\) | regression coefficient for covariate (β_x) |
| \(\regcoef\) | regression coefficient symbol β |
| \(\ve\) | residual vector e (ḛ) |
| \(\sMAEt\) | sample mean absolute error operator sMAE, the estimator of pMAE (no hat form is needed) |
| \(\sMSEt\) | sample mean squared error operator sMSE, the estimator of pMSE (no hat form is needed) |
| \(\sMSE{a}\) | sample mean squared error sMSE(·) applied to argument |
| \(\xbar\) | sample mean x̄ |
| \(\sRMSEt\) | sample root mean squared error operator sRMSE, the estimator of pRMSE (no hat form is needed) |
| \(\score\) | score function ℓ′ |
| \(\stprod{a}\) | self transpose-based product x′x |
| \(\set{a}\) | set notation {·} |
| \(\poolsize\) | side length q of a square pooling block, in pixels |
| \(\sign\) | sign operator |
| \(\sqf{a}\) | square of argument (·)² |
| \(\normsq{a}\) | squared norm ‖·‖² |
| \(\SE{a}\) | standard error function SE(·) |
| \(\ind\) | statistical independence symbol (⊥⊥) |
| \(\sumiN\) | sum over i from 1 to N |
| \(\sumin\) | sum over i from 1 to n |
| \(\sumip\) | sum over i from 1 to p |
| \(\sumjp\) | sum over j from 1 to p |
| \(\sumto{a}{b}\) | sum over the first argument from 1 to the second |
| \(\targetfn\) | target function \(\mu\): the true relationship between inputs and outcomes |
| \(\targetfnf{a}\) | target function applied to its argument, \(\mu\)(·) |
| \(\armijo\) | the Armijo sufficient-decrease constant of a backtracking line search |
| \(\degmat\) | the degree matrix of a similarity graph |
| \(\recallweight\) | the F-beta weight on recall |
| \(\schedfrac\) | the fraction of a learning-rate schedule that has elapsed |
| \(\vhidgrad\) | the gradient of the loss with respect to a hidden layer’s output |
| \(\shrinkage\) | the gradient-boosting shrinkage factor |
| \(\leak\) | the negative-side slope of a leaky ReLU |
| \(\svfrac\) | the nu-SVM bound on the fraction of margin errors |
| \(\schedlen\) | the number of updates a learning-rate schedule runs over |
| \(\linefn\) | the one-variable function a line search minimizes |
| \(\coordfactor\) | the per-coordinate contraction factor of gradient descent |
| \(\convrate\) | the per-step contraction factor of gradient descent |
| \(\prevalence\) | the prevalence of the positive class |
| \(\typetwoerr\) | the probability of a type II error |
| \(\siglevel\) | the significance level of a test |
| \(\tf\) | therefore symbol (∴) |
| \(\vt\) | time vector t lowercase (t̰) |
| \(\dif{a}{b}\) | total derivative fraction dF/dt (numerator first, denominator second) |
| \(\tp{a}\) | transpose of argument x′ |
| \(\truemean\) | true mean function \(\mu\): the population mean of the outcome given the inputs |
| \(\truemeanf{a}\) | true mean function applied to its argument, \(\mu\)(·) |
| \(\vY\) | uppercase outcome vector Y (Y̰) |
| \(\hidval\) | value of a hidden unit in a neural network, h |
| \(\hidvalof{a}\) | value of the hidden unit given as the argument, h with the unit as subscript |
| \(\vcdim\) | Vapnik-Chervonenkis dimension d_VC: the largest number of points the model family shatters |
| \(\Var{a}\) | variance function |
| \(\vcdimf{a}\) | VC dimension of a model family, d_VC(·) |
| \(\va\) | vector a (a̰) |
| \(\vg\) | vector g, such as a gradient (g̰) |
| \(\vecf{a}\) | vector notation |
| \(\vec{a}\) | vector notation (undertilde style) |
| \(\vhidval\) | vector of hidden-unit values (or a recurrent network’s hidden state), h̰ |
| \(\voutvar\) | vector of observed outcomes |
| \(\vone\) | vector of ones |
| \(\vprederr\) | vector of prediction errors, predicted minus observed outcomes |
| \(\vp\) | vector p, such as a probability distribution (p̰) |
| \(\vq\) | vector q, such as a second probability distribution (q̰) |
| \(\vr\) | vector r (r̰) |
| \(\v{a}\) | vector shorthand |
| \(\vu\) | vector u (ṵ) |
| \(\vv\) | vector v (v̰) |
| \(\vz\) | vector z (z̰) |
| \(\imgwidth\) | width of an image or feature map, in pixels |
| \(\vzero\) | zero vector |
References
[1]
G. James, D. Witten, T. Hastie, and R. Tibshirani, An introduction to statistical learning with applications in R, 2nd ed. Springer, 2021. Available: https://www.statlearning.com/
[2]
G. James, D. Witten, T. Hastie, R. Tibshirani, and J. Taylor, An introduction to statistical learning: With applications in Python. Springer, 2023. doi: 10.1007/978-3-031-38747-0.
[3]
T. Hastie, R. Tibshirani, and J. Friedman, The elements of statistical learning: Data mining, inference, and prediction, 2nd ed. Springer, 2009. Available: https://hastie.su.domains/ElemStatLearn/
[4]
Y. S. Abu-Mostafa, M. Magdon-Ismail, and H.-T. Lin, Learning from data: A short course. AMLBook, 2012. Available: https://amlbook.com/
[5]
C. M. Bishop, Pattern recognition and machine learning. in Information science and statistics. New York: Springer, 2006.
[6]
K. P. Murphy, Probabilistic machine learning: An introduction. in Adaptive computation and machine learning. MIT Press, 2022.
[7]
B. Hutchinson, “DATA 471/571: Machine learning.” Western Washington University, 2025.
[8]
K. D. Harris, “DATA 471: Machine learning, weekly lecture notes.” Western Washington University, 2022.
[9]
A. J. Dobson and A. G. Barnett, An introduction to generalized linear models, 4th ed. CRC press, 2018. doi: 10.1201/9781315182780.
[10]
P. K. Dunn and G. K. Smyth, Generalized linear models with examples in R, vol. 53. Springer, 2018. doi: 10.1007/978-1-4419-0118-7.
[11]
G. J. McLachlan and T. Krishnan, The EM algorithm and extensions, 2nd ed. John Wiley & Sons, 2007. doi: 10.1002/9780470191613.
[12]
S. N. Wood, Generalized additive models: An introduction with r, 2nd ed. Chapman; Hall/CRC, 2017. doi: 10.1201/9781315370279.