Notation Choices

This page records why the lab’s notes write certain symbols the way they do. Each entry states our convention, the reasons for it, and the other conventions you will encounter in other sources. Notes sites can link to an entry here instead of keeping their own copy.

The vector mark

Our convention is a tilde under the letter, as in \(\vx\). The shared macro \vecf (and shortcuts such as \vx and \vbeta) prints it.

We chose a mark underneath for three reasons.

Bold letters are hard to write by hand. When we work problems by hand, for example on a board, a bold letter is hard to copy neatly with chalk or a marker. A small tilde under a letter is quick to write, so the handwritten symbol and the printed symbol look the same.
The space above a letter is already used. In statistics, a hat over a symbol means “estimated value”. A vector mark above the letter would have to share that space with the hat, and the two marks would be crowded together.
A mark below the letter stays in place when we add a hat. The vector mark stays below and the estimate mark stays above, so each mark can be read on its own.

Example 1 (An estimated vector) Suppose a regression model has a coefficient vector \(\vbeta\) with \(p\) entries. Its estimate from data is written \(\hvb\), with the hat above the symbol and the vector mark below it. We read \(\hvb\) as “the estimated vector \(\vbeta\)”. Its \(j\)-th entry is the estimate of the \(j\)-th coefficient. That entry is a single number, so it needs no vector mark, only the hat.

Other sources use bold letters (\(\mathbf{x}\)) or arrows (\(\overrightarrow{x}\)) for the same objects. All of these notations mark a vector.

Subscripts and parentheses

Our convention splits the work between a subscript and parentheses. An indexed family is a collection of objects labeled by an index. A subscript names which member of an indexed family we mean. A subscript holds an index, such as the \(i\) in \(x_i\), or a parameter that picks one function or distribution out of a family, such as the \(\theta\) in \(f_\theta\). Parentheses hold the input a function is evaluated at, as in \(f(x)\). So \(f_\theta(x)\) reads “the member of the family labeled \(\theta\), evaluated at \(x\)”.

We chose this split for one reason. Each slot then answers one question about the symbol, so once you know the split, you can tell what each part of a symbol is doing.

Nothing forces the split. Sources divide the work differently, and you will encounter the same objects written the other way around. When you read a formula from another source, first work out what each subscript and each input in parentheses is doing: labeling a member of an indexed family, or being the input of a function. That tells you which convention the source follows, whatever its layout.

Example 2 (A stochastic process uses both slots) A random experiment is a process that involves chance, such as a coin flip, and an outcome \(\omega\) is one possible result of it. A random variable is a function whose input is an outcome \(\omega\). A stochastic process is an indexed family of random variables \(X_t\), one for each index \(t\), such as a time. The index goes in the subscript, because it picks a member of the family. Each \(X_t\) is itself a function of the outcome \(\omega\), so the outcome goes in the parentheses: \(X_t(\omega)\). Fixing \(\omega\) and letting \(t\) vary gives one ordinary function of \(t\), called a path of the process. Many sources drop \(\omega\) and write \(X_t\). Others write \(X(t)\), often when \(t\) is continuous, which puts the index in the parentheses instead of the subscript.

Binary labels

A binary classification problem has two classes, and each observation carries a label that names its class. We call the two classes the positive class and the negative class. When a formula does not need numerical labels, we write the classes as the sign symbols \(\posclass\) and \(\negclass\), from the macros \posclass and \negclass, and we use the same symbols for the event that a binary classifier or diagnostic test returns a positive or negative result. The confusion-matrix counts \(\ntp\), \(\nfp\), \(\nfn\) and \(\ntn\) use the same symbols in their subscripts. Those two sign symbols are class names rather than numbers, unlike the numerical labels \(+1\) and \(-1\) used in some models.

We write \(\negclass\) rather than \(\neg \posclass\) for the negative class or a negative test result because \(\neg\) is the operator for the complement of an event or the negation of a statement, and \(\posclass\) and \(\negclass\) are paired sign symbols. Writing \(\neg \posclass\) mixes those two conventions and reads as “not plus” rather than “negative”. When a binary classifier or diagnostic test assigns every observation to one of the two classes, the event \(\negclass\) of a negative result is the complement \(\neg \posclass\) of the event \(\posclass\) of a positive result, so the two are equal as events.

In medical diagnostic testing, other sources often write \(T^+\) and \(T^-\), or words such as “pos” and “neg”, for positive and negative test results.

When a model needs numerical labels for the two classes, our convention is to code the labels as \(0\) and \(1\), as in logistic regression. There the label \(1\) means the positive class and the label \(0\) means the negative class. The chapter on support vector machines switches to the numerical labels \(-1\) and \(+1\), and says so where it does.

We switch to \(-1\) and \(+1\) for support vector machines for one reason. A classifier outputs a number. It predicts the class \(+1\) when the number is positive, and the class \(-1\) when the number is negative. With \(\pm 1\) labels, a prediction is correct when the output and the label have the same sign. So the product of the label and the output is positive for a correct prediction and negative for a wrong one. The size of the product measures how far the output is from zero, on the correct side when the prediction is correct. With \(0\) and \(1\) labels, the same test needs two cases. If the output is a probability, a correct prediction is an output above \(0.5\) with the label \(1\), or an output below \(0.5\) with the label \(0\).

Both numerical codings (\(0/1\) and \(\pm 1\)) and the sign symbols (\(\posclass\) and \(\negclass\)) describe the same two classes. Other sources pick one coding and use it everywhere, so check which one a formula assumes before comparing it with ours.

The intercept

A linear model for an outcome adds up the inputs times their coefficients, and then adds one more number that does not multiply any input. That number is the intercept. Our convention is to write the intercept separately, as \(\coef{0}\). The coefficient vector \(\vcoef\) then holds the slopes only, meaning the coefficients that multiply the inputs. The input vector \(\vpredvar\) holds the inputs.

The number \(\coef{0} + \dprod{\vcoef}{\vpredvar}\) is called the linear predictor.

Writing the intercept separately keeps it visible as its own number. In logistic regression, for example, the logistic function turns the linear predictor into a probability. The intercept then shifts the decision boundary, the set of inputs where the model is undecided between the two classes, while the slopes tilt it.

Other sources absorb the intercept. They add a constant input \(1\) to the start of the input vector, and the coefficient vector then also holds the intercept, at its start. The linear predictor is then \(\dprod{\vcoef}{\vpredvar}\) alone. Both forms describe the same model. When a source writes the linear predictor as \(\dprod{\vcoef}{\vpredvar}\), check whether the input vector starts with a constant \(1\).

Counting the layers of a neural network

A neural network passes its input through a sequence of layers. Each layer is a row of units, and each unit computes one number. Between two neighboring layers, a weight matrix holds the numbers that say how much each unit in the first layer contributes to each unit in the second.

Suppose a network has an input layer, one hidden layer (a layer between the input and the output), and an output layer. Some sources call it a three-layer network, because it has three rows of units. Our convention is to call it a two-layer network. Here “two-layer” counts the weight matrices, not the rows of units. The network has two weight matrices: input to hidden, and hidden to output. We can also say that it has one hidden layer.

We chose to count the weight matrices because the input layer has no weights of its own and does no computation. Its values are the data. Each weight matrix is one computation from one row of units to the next, so counting the weight matrices counts the steps that do work. This is a choice, not a requirement, and both names are in use. Check which one a source means before comparing the depths of two networks.

Conventions for cross-entropy

Cross-entropy is a measure of how well a model’s predicted probabilities match the observed labels. For labeled data, it adds up the negative logarithms of the probabilities the model gave to the labels that were observed. A lower cross-entropy means a better match. Two choices change its numerical value without changing which of two models it prefers on the same data.

The first choice is the base of the logarithm. The base is free, so state it whenever you quote a number. Base \(e\) gives units called nats, and base \(2\) gives units called bits. Take a two-class model that gives probability \(0.5\) to each class. For a single example, its cross-entropy is \(\log_e 2 \approx 0.693\) nats, or \(\log_2 2 = 1\) bit. Changing the base multiplies every cross-entropy by the same positive constant, so it cannot change which of two classifiers you would pick. It does change the number, so a cross-entropy quoted without its base cannot be checked against a number from another source.

The second choice is whether to sum or to average over the examples. Our convention is to sum over all examples. Some software reports the average under the same name. On the same data, dividing every cross-entropy by the number of examples does not change which of two models is preferred, but it does change the number. So also check whether a source reports the sum or the average before comparing its numbers with ours.

Surprisal and the Kullback–Leibler divergence

The surprisal of an outcome \(x\) is written \(\surprisalf{x}\). The Kullback–Leibler divergence of \(q\) from \(p\) is written \(\kldivf{p}{q}\), with the true or reference distribution first. Our convention puts \(p\) first, because the divergence is the expectation under \(p\) of the log ratio of \(p\) to \(q\). It is not symmetric, so swapping the arguments changes the value. Some sources write the same quantity with the arguments in the other order, or call it the relative entropy, so check which distribution comes first before comparing a formula with ours.

The cross-entropy of \(q\) relative to \(p\) is written \(\crossentropyf{p}{q}\), again with \(p\) first, and it uses the same letter as the entropy because \(\crossentropyf{p}{p}\) is the entropy. It is not symmetric either, so check the order of the arguments in a source before comparing a formula with ours.

Names for the same object in machine learning and statistics

Machine learning and statistics texts often use different names and symbols for the same object. Each chapter of our notes says which symbols it uses. For example, the chapter on neural networks writes \(\vw\) and \(b\), and the regression chapters write \(\vcoef\) and \(\coef{0}\).

A linear function is written \(f(\vx) = \dprod{\vw}{\vx} + b\) in neural network texts. There \(\vw\) holds the weights, and \(b\) is the bias, which plays the role of the intercept. This “bias” is a different idea from the bias of an estimator in statistics. Statistics texts write the coefficients as \(\vbeta\) and the intercept as \(\beta_0\), as in the intercept entry. The two notations describe the same linear model.
Machine learning texts often write \(D\) or \(d\) for the number of inputs and \(N\) for the number of observations. Statistics texts write \(n\) for the number of observations and \(p\) for the number of inputs, which they call predictors.
Neural network and pattern recognition texts often call the label the target. Statistics texts call it the response or the outcome. In classification, all of these names mean the same object as our label.

You will read both literatures, so expect to encounter both sets of names and symbols.

Writing the likelihood function

The likelihood function is a function of the model’s parameters, with the observed data held fixed. Our convention is mostly \(\Lik(\theta)\), which leaves the data implicit. We use a form that shows the data, such as \(\Lik(\theta; \vx)\), when we want to name which data set the likelihood is computed from. We chose \(\Lik(\theta)\) to emphasize that the likelihood is a function of the parameters.

Other sources write the same likelihood in other ways:

\(\Lik(\vx; \theta)\).
\(\Lik_{\vx}(\theta)\).
\(\Lik(\vx \mid \theta)\) or \(\Lik_{\theta}(\vx)\).

We read every one of these forms as the likelihood: \(\vx\) is the observed data, held fixed, and \(\theta\) is the input that varies. The forms \(\Lik(\vx \mid \theta)\) and \(\Lik_{\theta}(\vx)\) write the data where a formula for the distribution of the data would write it. That formula is a function of the data, with the parameter held fixed. The likelihood uses the same formula, but treats the parameter as the input and holds the data fixed. So when a source writes one of these two forms, check that it means a function of \(\theta\).

Writing the score function

The log-likelihood is the natural logarithm of the likelihood function. The score function is the derivative of the log-likelihood with respect to the parameter. This is the score of statistics, which is not the linear predictor of the intercept entry. When the model has several parameters, the score is a vector that holds one partial derivative for each parameter, taken with the other parameters held fixed. We often omit its arguments and write \(\score\) for \(\score(\theta)\). When we want to show the data, we write \(\score(\theta; \vx)\), in the same style as the likelihood.

Some sources write \(U\) or \(S\) for the score function instead. We write \(\score\) because it keeps \(U\) and \(S\) free for other uses.

A partial derivative written with a slash

A partial derivative can be written as a stacked fraction, \(\derivf{f}{x}\), or with a slash, \(\derivfslash{f}{x}\). The two mean the same thing. We write \derivf{f}{x} in displayed equations and \derivfslash{f}{x} in running text, where a stacked fraction would push the lines of a paragraph apart. Use the macro rather than typing \partial f / \partial x, so a reader of the source sees one notation for one object. Put the slash form inside \paren{} before a subscript or an exponent: \(\derivfslash{f}{x}_i\) reads as a derivative with respect to \(x_i\), and \(\paren{\derivfslash{f}{x}}_i\) does not.

Other sources also write \(f_x\) or \(\partial_x f\) for the same derivative. We do not use them, because our subscripts mark indices, as in \(x_i\) for the \(i\)th observation.

Symbols for estimands and estimators

An estimand is the quantity we want to learn about, and an estimator is a rule that computes a guess of it from data.

Our convention is to write estimands with lower-case Greek letters, such as \(\alpha, \beta, \gamma, \delta\). We write an estimator by placing a hat over the symbol of its estimand, as in \(\hat\theta\). Usually the dependence of the estimator on the data is left implicit.

In the source, we write the hat with the macro \est{\theta}, which prints \(\est{\theta}\), or with a named macro that begins with e, such as \eth or \ecoef{x}. The older hat macros that begin with h, such as \hth and \hcoef{x}, have e counterparts that print the same symbols. Most of them are marked “deprecated, use” with the replacement in interpretations.tsv. We made this decision on 2026-10-22, so that a hat has one source form, the e macros, which name the estimate they print. Each deprecated macro still works, so no existing page breaks.

Other sources use other symbols, so check which symbol a source uses for the estimand before reading a formula.

Short symbols for the pieces of the bias-variance decomposition

Fix a covariate value \(\predvar\), a training sample size, and a fitting procedure. The fitted prediction function \(\epredfn\) varies from one training data set to the next. We follow ISL: the prediction function is \(f\), the fitted one is \(\epredfn\) (from the macro \epredfn), and the fitted value at a point is \(\eoutvar\), which prints \(\hat{y}\). The macro used to print \(g\), so the fitted function printed as \(\hat{g}\). On 2026-10-22 we chose \(f\) provisionally, because it is the symbol in the textbooks. On 2026-10-23 we chose to follow ISL’s notation, after a request to write the function as \(\hat{y}\) was dropped, so the function is \(f\) and \(\hat{y}\) stays the fitted value. ESL also writes the fitted function as \(\hat{f}\), and uses \(\hat{G}\) for a predicted class. The older macros \hf and \hfx{x} print the same fitted function and are deprecated in favour of \epredfn and \epredfnf{x}. Let \(\Outvar\) be a new outcome at \(\predvar\), independent of the training data. Write \(\truemeanf{\predvar}\) for the mean of \(\Outvar\). The noise is the random part of \(\Outvar\) that no function of the covariates predicts. Three short symbols name the pieces of the expected squared error in predicting \(\Outvar\):

The bias of the fitted value at \(\predvar\) is \(\biassymf{\epredfnf{\predvar}}\), from the macro \biassymf. It is the mean of the fitted value over training data sets, minus the true mean \(\truemeanf{\predvar}\). This is the expected error of the fitted value as an estimate of the true mean. The long operator \(\Biast\) stays available for the full definition.
The variance of the fitted value over training data sets is \(\varsymf{\epredfnf{\predvar}}\), from the macro \varsymf. The macro \varsymf is a variance operator for any random variable, in the style of the expectation \(\E{\cdot}\).
The residual variance is \(\resvar\), from the macro \resvar. It is the variance of the noise given the inputs, so it is the existing variance parameter \(\varpar\). When the noise variance depends on \(\predvar\), the residual variance is a function of \(\predvar\). Many sources call it the irreducible error. We avoid that name because the residual variance is a variance and not an error, and because it is irreducible only given the inputs: a further input that predicts the outcome can reduce the average residual variance. The macro \irrerr stays available as a deprecated alias.

With these symbols, the expected squared error splits into three pieces, where the expectation is over the training data and over \(\Outvar\):

\[ \E{\sqf{\Outvar - \epredfnf{\predvar}}} = \sqf{\biassymf{\epredfnf{\predvar}}} + \varsymf{\epredfnf{\predvar}} + \resvar. \]

A derivation that does not manipulate a piece can write its short symbol, after the piece is defined in full. Each piece is a population quantity. It is a fixed number once \(\predvar\), the population, the fitting procedure and the training sample size are fixed.

We write the expectation, the probability, the bias and the variance as operators in blackboard bold: \(\Expp\), \(\P\), \(\biassym\) and \(\varsym\). The blackboard letter marks an operator that takes a random variable or an estimator, so it cannot be mistaken for an italic variable. We do not use blackboard bold for the risk, because \(\reals\) is the real numbers. The risk stays an italic \(\poprisk\), which is free now that the noise is \(\varepsilon\). The long operators \(\Biast\) and \(\Vart\) stay upright words. We make this exception to the Greek convention for estimands in the estimands entry. The variance operator applies to any random variable, not only to a fitted value. The bias operator is applied to an estimator, and its letter matches the name of the long operator \(\Biast\). Like the expectation, each has two macros. The macros \biassym and \varsym print the bare operators \(\biassym\) and \(\varsym\), as \Expp prints \(\Expp\). The macros \biassymf and \varsymf apply the operator to an argument in square brackets, as \E does.

These Latin letters have collisions:

The macros \Regcoef and \Coef print an italic \(\Regcoef\), for regression coefficients, which are variables. The bias operator prints a blackboard bold letter, so the two are distinguishable in one document.
The existing macro \V prints \(\V\), the full name of the variance, so \varsymf is the short form, not a replacement.

The residual variance keeps a Greek letter, because it is a variance parameter and not an operator. It reuses \(\sigma^2\), which already names the variance parameter, instead of getting a new letter.

Some sources write the same pieces in other ways:

They keep the operator form, as in \(\Biast(\hat f(x))^2\) and \(\Vart(\hat f(x))\), in every line of the derivation. This is unambiguous but long.
They write \(\sigma_\epsilon^2\) or \(\Vart(\epsilon)\) for the residual variance, where \(\epsilon\) is their noise symbol.
They may use a lower-case Latin letter for each piece. We avoid this because \(b\) already names the bias of a neural network in the names entry, and the macro \vv prints the vector \(\vv\).

Check which form a source uses before comparing its decomposition with ours.

One symbol for a variance

We looked for a single symbol that could replace \(\varpar\), the variance parameter, and we found none. On 2026-10-26 we chose to keep \(\varpar\) for the parameter and \(\varsym\) for the variance of a random variable. We add no macro and no new Greek letter.

We checked these books, which Notation Across Sources lists in full:

ISL, 2nd ed. (eq. 3.8, p. 77), writes \(\sigma^2 = \Vart(\epsilon)\). ESL (eq. 2.27, p. 26) writes the variance parameter as \(\sigma^2\), and Bishop (Section 1.2.4, p. 24) does too.
Bishop (eq. 1.40) writes the variance of a random variable as lower-case \(\mathrm{var}[f]\). Murphy, Probabilistic Machine Learning (eq. 2.29), writes it as \(V[X]\), with a Latin capital V, and writes the precision as \(\lambda = 1/\sigma^2\).
Dobson and Barnett, 4th ed. (Notation, p. 7), writes \(\mathrm{var}(Y_i) = \sigma_i^2\).

No book we checked uses one glyph for the variance parameter. The nearest single letter is the operator \(\varsym\), which we already have.

We rejected these letters, each for a collision in the macros:

\(\phi\) and \(\varphi\) are the dispersion parameter of a generalized linear model. Dobson and Barnett, 4th ed. (overdispersion paragraph for binomial data, PDF page 184), writes \(\mathrm{var}(Y_i) = n_i \pi_i (1 - \pi_i) \phi\) for overdispersed binomial data, so \(\phi\) multiplies the variance and is not the variance. The macros \normpdf, \linefn and \poismean also print \(\phi\).
\(\nu\) prints a shrinkage factor (\shrinkage), a bound on the fraction of margin errors (\svfrac), a second measure (\measureb) and a nullity (\nullity).
\(\omega\) prints an outcome (\outcome).
\(\vartheta\) looks like \(\theta\), which names a parameter.
\(\varrho\) looks like \(\rho\), which is the overdispersion parameter \overdisp.
\(\varpi\) looks like \(\pi\).
\(\varsigma\) is the only unused letter that is a form of \(\sigma\). We rejected it because it looks like \(\zeta\) or \(c\) at a small size and is hard to write by hand.
A lower-case Latin \(v\) collides with the vector macros, such as \vv.
A capital \(V\) collides with \V and with the variance function \(V(\mu)\) of a generalized linear model.

The precision \(\tau\) (\precision) stays a separate symbol, because it is the reciprocal of the variance.

Noise, deviation and error

The noise is a deviation. An estimation error and a prediction error are errors, which are deviations whose reference is an estimand. Each of the three has its own reference.

Our convention is to write a deviation as \(\devof{a}{b}\), the quantity \(a\) minus the reference value \(b\). The noise is the deviation of an outcome from its mean given the input, \(\Noise = \devof{\Outvar}{\truemeanf{\predvar}}\). An estimation error is the deviation of an estimate from its estimand, \(\devof{\est{\theta}}{\theta}\), and we write it \(\esterr{\est{\theta}}{\theta}\). A prediction error is the estimation error whose estimand is the new outcome. In meaning, these three differ only in the reference. The macro \devof states that shared structure, and prints the difference itself. The macro \esterr prints the error symbol \(\varepsilon\), with the estimand as a subscript, so an error is not mistaken for another deviation. The macros \Noise and \prederr have their own symbols, \(\Noise\) and \(\prederr\). In all three we subtract the reference from the quantity. So an estimate or prediction that is too large has a positive error. The residual \resid is the exception, because it is the observed outcome minus the predicted outcome, the opposite sign of the prediction error. The macro \devof always prints parentheses around the difference, so a power such as \(\devof{a}{b}^2\) squares the whole difference. When the reference is a sum, wrap it in \paren yourself, as in \(\devof{a}{\paren{b + c}}\). The macro \devsq prints a squared deviation, and \ssdev prints the sum of squared deviations over observations. Both print the operator \(\devsym\) and not a difference.

We follow the textbooks An Introduction to Statistical Learning (ISL) and The Elements of Statistical Learning (ESL) and write the noise as the Greek letter \(\varepsilon\). The macros \Noise and \noise both print \(\Noise\), so a random noise variable and its realized value share one symbol, as in those books. The vector of noise variables and the vector of realized values are both \(\vNoise\). The noise differs from one observation to the next and has a distribution. In real data we never observe a realized noise value, but we write it when we reason about a single observation. This is an exception to our rule that a random variable gets a capital Latin letter and a Greek letter names a parameter. We accept the exception because every common textbook uses \(\varepsilon\) for the noise.

Strictly, the noise is a deviation and not an error. It is the outcome minus its mean given the input, so nothing was estimated and nothing went wrong. An estimation error is an estimate minus its estimand, and a prediction error is the case where the estimand is the new outcome. The textbooks call \(\varepsilon\) the error term, but we call it the noise.

We keep the errors distinct from the noise by giving them a label. The macro \esterr{a}{b} prints \(\esterr{a}{b}\), the error of the estimate \(a\) for the estimand \(b\). The macro \prederr prints \(\prederr\), with the upright label “pred”. The macro \prederrf{g} prints \(\prederrf{g}\), the prediction error of the predictor \(g\). It is the short form of the estimation error of \(g\) for the new outcome, \esterr{g}{\Outvar}, and uses the label “pred” in place of the estimand. The older macros \err and \eps print the bare \(\varepsilon\), the same symbol as the noise.

The residual is computed from a fitted model and so is observed. It estimates the noise, with the same sign: both are an outcome minus a reference. So we write the residual as the estimate of the noise, \(\resid\), using \est as we do for every other estimate. This needs no new letter. ISL writes the residual as \(e_i = y_i - \hat{y}_i\), but the letter \(e\) is Euler’s number (\euler, \e{}) and the start of the expectation operator \(\E{\cdot}\) and its estimate, so we avoid it. The standardized residual is \(\stdresid\) and the modified residual is \(\modresid\). The residual has no separate capital: \Resid prints the same \(\resid\).

Earlier versions of these macros printed the noise as \(U\) with realized value \(u\), and before that as the Greek letter \(\nu\). Any use that meant \(\nu\) or expected \(U\) or \(u\) needs a sweep. The macros \popdev and \cdev now print \(\deviation\), the observed deviation \devn, for a deviation from a mean, where they used to print \(\nu\). The macro \deviationf prints the operator \(\devsym\) with the reference as a subscript, for a labelled deviation of a given value. We keep \popdev and \deviationf for existing uses. A deviation has two forms, as an expectation does. The operator that takes a value to its deviation from a reference is uppercase blackboard bold, \devsym, which prints \(\devsym\), like \(\P\) and \(\varsym\). An observed deviation is the lowercase letter, \devval or \deviation, which prints \(\deviation\). Before this change both printed \(\delta\). In a PDF the lowercase letter needs the bbm package, which macros-pdf-header.tex loads, so list that file before macros.qmd in include-in-header. The design matrix \design is an italic \(\design\), so the operator does not collide with it.

A sample mean is a bar over the quantity, and we write it with the macro \smean{y}, which prints \(\smean{y}\), rather than with a literal \bar. The macros \xbar, \outbar, \predbar, \rvarbar, \samplemean and \avgleverage use it. The sample mean of the deviations from a population mean is the deviation of the sample mean, \(\devof{\smean{y}}{\mu}\). The macro \smeandev prints it as \(\smeandev\). Other uses of a bar, such as \Ebar and \incidence, are not sample means and keep their own macros.

Some sources use \(e\) for the noise, so check what a source means by it. Some sources subtract in the other order, so check the sign before comparing a formula with ours.

A deviation and a difference

A deviation and a difference are both one value minus another. They differ in what the second value is.

A deviation is a value minus a fixed reference, such as a mean or an estimand. We write it with \(\deviation\), or with \devof for the explicit form.
A difference compares two values of equal standing, such as two input vectors or the log-odds at two inputs. We write it with \(\diff\), as in \diffvx for the inputs and \difflogodds for the log-odds.

We keep two symbols because the two ideas answer different questions. A deviation answers “how far is this from where it should be?”. A difference answers “how much does this change between two values?”. The deviance of a fitted model is a difference, between the minus twice log-likelihood at the fitted model and at the saturated model, so it is named \(\deviance\) rather than written with either symbol.

The word “discrepancy” is not a defined term here. In prose, use the defined term that fits: a deviation, a difference, an error or a residual. The phrase “low-discrepancy sequence” is a standard name, so we keep it as it is.

Italic and upright

We use italic and upright type to carry meaning. An italic letter is a variable, a value or a parameter. This includes the Greek letters, such as \(\varepsilon\), \(\mu\) and \(\theta\), and an estimate such as \(\resid\). An upright letter or word is a name, not a quantity. It is an operator, a function name, a constant or a label, such as \(\tr\), \(\diag\), \(\rank\), the constant \(\euler\) and a text subscript like \(\vcdim\). So we never print a variable upright, and we never print an operator or a constant in italic. For example, \Regcoef prints an italic \(\Regcoef\), because a regression coefficient is a variable. The degrees of freedom \df prints the upright name \(\df\) instead of \(\nu\), because an abbreviation is a name. That leaves \(\nu\) to the macros that still print it, such as \shrinkage.

We follow ISO 80000-2:2019, the international standard for mathematical notation, on this split. It sets variables and generic function symbols, such as \(f\), in italic, and the names of specific functions and operators, such as \(\sin\) and \(\exp\), upright. So the prediction function \predfn (\(\predfn\)), its estimate \epredfn (\(\epredfn\)) and the best prediction function \bestpredfn (\(\bestpredfn\)) stay italic, because each is a generic function symbol. The predicted value \predvalf{x} (\(\predvalf{x}\)) stays italic too, because it is a value. It does not matter that the estimated function and the predicted value depend on the training data and so are random. Italic marks a symbol that stands for something that varies, and random quantities are among those. The operators that act on them, such as the expectation and the variance, are named, so they are upright or blackboard. The textbook ISL also writes \(f\) and \(\hat{f}\) in italic.

The design matrix and the design vector

The design matrix holds the input values for a model, one row for each observation, after any feature map. A feature map turns one input into a vector of features, such as its powers or the outputs of a layer of hidden units. Our convention is to write the design matrix as \(\design\), and the design vector for one input as \(\vdesign\). The vector mark is the same one that every vector here carries. Row \(i\) of the design matrix is the design vector of observation \(i\), transposed. The feature map itself, \featmap, prints the same letter, \(\featmap\).

We chose the letter \(D\) because the design matrix involves no parameters, so it has no reason to be a Greek letter. Greek letters are for parameters. Earlier versions used \(\phi\) for the feature map and the design vector; the design matrix was already \(D\). The macro \vfeatmap still works and prints the same symbol as \vdesign, so existing pages keep rendering. The letters \(\phi\) and \(\Phi\) stay with the standard normal density and distribution function, \normpdf and \normcdf.

The deviance used to print \(D\) as well, which collided with the design matrix. The deviance is twice the gap between the log-likelihood of the saturated model and that of the fitted model. It is a scaled statistic, not a deviation of a value from a reference, so it does not take the deviation symbol. \deviance now prints the upright name \(\deviance\), as the other named quantities do.

The deviance information criterion uses a different quantity, minus twice the log-likelihood with no saturated term. It has its own macro, \dicdev, which prints \(\dicdev\), and a call form \dicdevf{\vth}, so the two are not confused. The two differ by a constant that does not depend on the parameter, and that constant drops out when models are compared. In symbols, the deviance of a fitted model is the difference between this quantity at the fitted model and at the saturated model.

Other sources differ. Some write the design matrix as \(X\), and some write the feature map as \(\phi\) or \(\Phi\). Check which letter a source uses for each of the three objects before you compare formulas.

Probability notation

A random variable is a function of the outcome of a random experiment, as in the stochastic process example. Its value depends on chance. Our convention is to write random variables with uppercase letters, such as \(X\), \(Y\) and \(Z\), and their observed values with the matching lowercase letters, such as \(x\), \(y\) and \(z\). Some sources keep uppercase letters for matrices, and use other symbols for random variables. Check a source’s own notation section before assuming ours.

An event is something that may or may not happen, such as \(X > 2\). A probability mass function gives the probability of each value of a discrete random variable, one that takes separate values such as \(0, 1, 2\). A probability density describes a continuous random variable, one that takes values on a continuous range, such as a height.

We write \(\Pr()\) for the probability of an event, and \(\p()\) for a probability mass function or a probability density. Here \(\p\) is a function name, not the number of inputs. Some sources write \(P()\) for all of these. Others keep \(f()\) for densities and mass functions, and use \(P()\) only for the probability of an event.

When the distribution depends on a parameter, we write the parameter as a subscript, \(\Prunder{\theta}{A}\), read “the probability of \(A\) when the parameter is \(\theta\)”. The macro \Prunder takes the subscript first and the event second, so a source does not need to put a subscript between \Pr and its parentheses by hand. Some sources write \(P(A \mid \theta)\), which reads as conditioning on a random \(\theta\). We reserve the bar for conditioning on an event or a random variable, and use the subscript when the parameter is fixed rather than random.

We write the expectation of a random variable with square brackets, as in \(\E{X}\). Some sources write plain \(E[X]\), or parentheses, \(E(X)\). The meaning is the same.

We write \(X \ind Y\) for “\(X\) is independent of \(Y\)”. Some sources write \(X \perp Y\), or state independence only in words.

We write \(\neg A\) for the complement of an event \(A\), read “not \(A\)”. Some sources write \(A^c\) or \(\bar{A}\).

We write \(\eqdef\) for an equality that holds by definition. This tells it apart from an equality that follows from other facts. Some sources use a bare \(=\) for both.

We write \(\eqby{\text{(A1)}}\) for an equality that holds for the reason written over the sign. The reason can be an assumption, a numbered result or a rule, such as \(\eqby{\text{Thm 2}}\). This tells a reader which fact each step of a derivation uses. Some sources put the reason in words after the line, or leave it out. Equal by definition, \(\eqdef\), is the special case where the reason is the definition.

The true mean function and its conditional form

The true mean function gives the population mean of the outcome \(\Outvar\) at each value \(x\) of the predictor. Our convention is to write it \(\truemean\), and its value at \(x\) as \(\truemeanf{x}\). Both print \(\mu\).

When the outcome and the predictor need to be named, we write \(\truemeansubf{\Outvar \given \Predvar}{x}\). This prints \(\mu_{Y|X}(x)\), a shorthand for the conditional population mean \(\condE{\Outvar}{\Predvar = x}\).

Example 3 (The conditional population mean) Let \(\Outvar\) be a person’s height and \(\Predvar\) their age. Then \(\truemeansubf{\Outvar \given \Predvar}{10}\) is the population mean height of 10-year-olds. It is the same number as \(\condE{\Outvar}{\Predvar = 10}\).

The usual case has the outcome \(\Outvar\) and the predictor \(\Predvar\), and there we shorten that form. We write \(\cmean\) for the conditional population mean function. We write \(\cmeanf{x}\) for its value at \(x\). The macro \cmeanf{x} prints the same as \truemeansubf{\Outvar \given \Predvar}{x}. The macro \cmean prints \(\truemean\) with the subscript \(\Outvar \given \Predvar\), and no argument. So \(\cmeanf{10}\) is the same number as \(\condE{\Outvar}{\Predvar = 10}\). We keep \truemeansubf for any other pair of variables.

These four forms differ in how much they say.

\condE{\Outvar}{\Predvar = x} writes out the expectation, so the definition appears in the text.
\cmeanf{x} is the function notation for that expectation, used once the function is the object of study, as in “\(\cmeanf{x}\) is linear in \(x\)”.
\cmean names the function itself, without a value, as in “the function \(\cmean\)”.
\truemeansubf{A \given B}{x} is the general form, for variables other than \(\Outvar\) and \(\Predvar\).

\cmean and \cmeanf only fix the subscript of the subscripted \(\truemean\), so they add no new meaning. We defined them because \(\Outvar \given \Predvar\) is the subscript of the usual case. That subscript is long to write out each time. The c in each name stands for conditional, as the cond in \condE does. The f ending marks the applied form, as in \truemeanf.

For the general form we chose a separate macro, \truemeansubf, not an optional subscript on \truemeanf, for three reasons.

An optional argument breaks the lds site’s build. LaTeX and MathJax allow an optional argument only as the first argument, so it would be written \truemeanf[Y|X]{x}. The build_math_macros.py script that lds runs before rendering cannot read an optional argument with a default value. It would stop every page in every format after lds updated its latex-macros pin.
A separate macro works with the existing parsers. Its two arguments are both required, and \condE already has the same form.
The macro name tells a reader of the source that a subscript is present.

Other sources write the true mean function as \(\mu(x)\) or \(f(x)\), or write the conditional population mean in full as \(E[Y \mid X = x]\). The subscripted form \(\mu_{Y|X}(x)\) is also common.

We keep \popmean and \pmean as aliases of \mean, the population mean of a variable. We do not make them aliases of \truemean, and we do not define a \popmeanf as an alias of \truemeanf. In the source, \mean, \popmean and \pmean name the population mean of a variable. The names \truemean and \truemeanf name the true mean function. Each name then says which of the two a formula means, even though all of them print \(\mu\).

The logistic function

The logistic function maps a real number \(a\) to \(1 / (1 + e^{-a})\). Our convention is to print it as the upright name \(\sigmoidf{a}\), which is expit, with the inverse \(\logit\), the log-odds function. \expit is the primary name, and \sigmoid and \sigmoidf are aliases that print the same.

We chose this for two reasons. First, the symbol \(\sigma\) is the usual choice in the deep learning literature, but here it also prints a standard deviation, a kernel bandwidth, a singular value and the generator of a \(\sigma\)-algebra. Second, a named function is upright (ISO 80000-2), and expit is already a name in software: scipy calls it expit, and R calls it plogis.

Other sources use other forms. An Introduction to Statistical Learning writes it out as \(e^x / (1 + e^x)\). Deep learning texts write \(\sigma\) and call it the “logistic sigmoid”. We did not choose an upright \(\mathrm{s}\), which is easily read as the variable \(s\), or the text “sigmoid”, which is long inside network equations and also names \(\tanh\). In a neural network, where the activation function is a role rather than one function, a chapter can name the activation with a generic symbol and say which function it is.

We keep the \sigmoid and \sigmoidf aliases permanently rather than renaming every use, because a rename changes no output and the name sigmoid remains a natural way to say “the logistic function” in neural-network text.

An exponent on a function with built-in delimiters

Some macros print a function name with its argument inside delimiters, such as \Pf{x} (parentheses), \E{x} (square brackets) and \sigmoidf{x}. Our convention is that such a macro wraps its whole output in one pair of braces, as in \providecommand{\Pf}[1]{{\P \paren{#1}}}. An exponent written after the macro then applies to the whole expression, and it sits above the top of a tall argument, as in \(\Pf{\frac{a}{b}}^2\).

We chose this for one reason. The delimiter macros (\paren, \sb, \cb and similar) end in an empty closing atom, and an exponent that follows without the braces attaches to that empty atom. It then stays at the height of the text line. The braces change what is printed in one small way: the thin space between a leading digit and the function name is lost, as it already was for \lnf and \maxf.

A macro with no argument cannot be wrapped this way, so the aliases \ceil, \set and \strlen take an argument, as in \ceil{x}.

Another convention is to leave the macros unbraced and to wrap each use in braces. Every use that forgot the braces would then print the exponent at the wrong height.

Counting: cardinality, factorials and binomial coefficients

Our convention has three parts.

We write the number of elements of a finite set \(A\) as \(\card{A}\), with vertical bars.
We write the factorial of \(n\) as \(\factorial{n}\).
We write the number of ways to choose \(k\) of \(n\) objects as \(\binomial{n}{k}\), read “\(n\) choose \(k\)”.

The macros \card, \factorial and \binomial print them.

We chose these for three reasons.

\card names a role, so a reader of the source can tell a count of elements from an absolute value or a length, even though \card prints the same bars as \abs.
\binomial prints parentheses around the stacked pair, which distinguish \(\binomial{n}{k}\) from the fraction \(n\) over \(k\).
\factorial groups its argument and the exclamation mark, so an exponent written after it applies to the whole factorial, as in \(\factorial{n}^2\).

The \factorial macro puts an exclamation mark after its argument without adding parentheses. An argument with more than one symbol needs its own parentheses, as in \factorial{(n-1)}, which prints \(\factorial{(n-1)}\). Without them, \factorial{n-1} prints \(\factorial{n-1}\), which reads as \(n - (1!)\).

Other sources write the cardinality as \(\#A\) or \(n(A)\). They write the binomial coefficient as \(C(n, k)\) or \({}^{n}C_{k}\). When you read a formula from another source, check which of these a symbol means before you compare it with ours.

The power set

We write the power set of a set \(A\), the set of all its subsets, as \(\powerset{A}\), with a calligraphic P. The macro \powerset prints it, and \powersetsym prints the symbol alone. \powerset names the role, so a reader of the source can tell the power set from the other calligraphic P in these macros, \distfam, a family of distributions. A set with \(n\) elements has \(2^n\) subsets, which is why other sources write the power set as \(2^A\). Some write it with a plain \(P(A)\), or with Weierstrass’s \(\wp(A)\).

The sum of the elements of a vector

Our convention is to write the sum of the elements of a vector \(\vy\) as \(\sumof{\vy}\), which prints \(s_{\vy}\). We write the sum of the elements of a vector of random variables \(\vY\) as \(\Sumof{\vY}\), which prints \(S_{\vY}\). This follows the rule that a capital letter is a random variable and a lowercase letter is a value.

The two symbols are \sumsym and \Sumsym. The macros \sumof and \Sumof put the vector in the subscript, so a formula about a sum names the vector it sums.

We chose this for three reasons.

One symbol replaces a sum \(\sum_{i=1}^{n}\) whose index and limits would repeat in every line of a derivation.
The subscript names the vector, so the symbol stays unambiguous when two vectors appear in the same line.
The sum of the entries of \(\vy\) equals the dot product \(\vone \cdot \vy\) of \(\vy\) with the ones vector, so the symbol and the dot product are two names for one quantity.

Some sources write the sum of \(n\) random variables as \(S_n\). The subscript \(n\) there is the sample size, not the vector. Within these notes, \(S\) with two subscripts, such as \sspred, is a centered sum of squares or cross-products, not a sum of elements. When you read a formula from another source, check whether the subscript of \(S\) is a count or a vector before you compare it with ours.

The number of inputs and the number of input variables

An input is one observation’s vector of values, so there are \(\nobs\) inputs. An input variable is one coordinate of that vector, so there are \(\ninputvars\) input variables. The data matrix \(\mX\) has \(\nobs\) rows and \(\ninputvars\) columns. We name the count \ninputvars, not \ninputs, because “the number of inputs” can be read as either count. The older macro \inputdim is kept as an alias for \ninputvars.

Many sources write the number of input variables as \(p\) (ISL and ESL) or \(d\), and some write \(D\). We avoid \(D\) because it is the design matrix.

Sums of squares, and counts and indices for training

We write the residual sum of squares as \(\RSS\), from the macro \RSS, and the total sum of squares as \(\TSS\), from the macro \TSS. They are names of statistics, so they are upright, and a subscript marks a model, as in \(\RSS_0\). We avoid writing \text{RSS} in the source, because the macro keeps the spelling and the font the same on every page.

The training symbols below name a role, so the macro names the role and the letter is our choice.

The number of parameters in a neural network is \(\nnetparams\), from \nnetparams. We use a capital letter because it counts every weight and bias in the network, and it differs from the number of input variables \(\ninputvars\).
The index of a step of an iterative algorithm, such as gradient descent, is \(\iter\), from \iter.
The index of a layer of a neural network is \(\layeridx\), from \layeridx. The letter is the usual one for a layer index in deep learning, and it differs from the letter of the number of cross-validation folds \(\nfolds\). The same letter also names the log-likelihood \llik, the loss of one observation \obsloss and the norm \lpnorm. A layer index labels layers, not observations or parameters, so we accept the overlap rather than take a letter that is free but unusual.
The objective function that an optimization algorithm lowers is \(\objfn\), from the existing macro \objfn, and \objfnf{\vth} prints \(\objfnf{\vth}\). Lecture exercises that write a plain \(J\) for the cost use \objfn, so there is one macro for one concept.

Some existing macros already cover other symbols that lecture exercises write as a bare letter.

The number of models in a finite family is \nmodels, which prints \(\nmodels\).
The number of cross-validation folds is \nfolds, which prints \(\nfolds\), and the index of a fold is \ifold.
The number of features in a feature vector is \ninputvars, which prints \(\ninputvars\), because we avoid \(D\) for a count. See the input counts entry.

We do not give the size of a minibatch a letter of its own. Write it as the cardinality of the set of indices, such as \card{B_t}, because the letter \(b\) is already the bias of a neural network in the names entry.

Exponential families

We write the density of a one-parameter exponential family in its natural form, \(\basemeasuref{x} \expf{\natparam \statf{x} - \logpartitionf{\natparam}}\). Each symbol is a macro that names a role, so the letter is our choice.

The natural parameter is \(\natparam\), from \natparam, and the vector form is \(\vnatparam\), from \vnatparam.
The sufficient statistic is \(\stat\), from the existing macro \stat, and \statf{x} prints \(\statf{x}\).
The base measure is \(\basemeasure\), from \basemeasure, and \basemeasuref{x} prints \(\basemeasuref{x}\). It is the same factor as the existing \rfac in the factorization theorem, so \basemeasure expands to \rfac, and the two always print alike. The letter \(h\) also names the hazard function \h; the argument tells them apart, since the hazard takes a time.
The log-partition function is \(\logpartition\), from \logpartition, and \logpartitionf{\natparam} prints \(\logpartitionf{\natparam}\).

The natural parameter prints the same \(\eta\) as the linear predictor \lincomp of a generalized linear model, and the two are the same quantity only for the canonical link. We keep two macros so that a page can say which one it means.

Other sources use other letters for the same roles. Casella and Berger write \(h(x)\,c(\theta)\exp\paren{\sum_{i=1}^k w_i(\theta) t_i(x)}\) with \(k\) terms; we write the case \(k = 1\). Their normalizing constant \(c(\theta)\) is \(\expf{-\logpartition}\) when the single weight \(w(\theta)\) is the natural parameter, and their \(h(x)\) is our base measure. Dobson and Barnett write \(\exp\paren{a(y) b(\theta) + c(\theta) + d(y)}\). When \(a(y) = y\), their \(b(\theta)\) is the natural parameter, \(c(\theta)\) is minus the log-partition function, and \(\exp\paren{d(y)}\) is the base measure.

Level sets and preimages

A level set of a function \(f\) at a value \(c\) is the preimage of the single value \(c\), so we write it as a preimage. The macro \preimage{f}{S} prints \(\preimage{f}{S}\), the set of inputs whose values are in \(S\), and \levelsetof{c}{f} prints \(\levelsetof{c}{f}\).

We chose this form for two reasons.

It defines the symbol by its most literal meaning, and the level set follows as a special case with \(S = \cb{c}\).
It does not reuse \(\mathcal{L}\), which \Lik already prints for the likelihood.

The macro \levelset, which printed the script letter \(\mathcal{L}\), is removed.

The endpoints of a confidence interval

Our convention is to call the lower and upper endpoints of a confidence interval \(\cilo\) and \(\ciup\), which are statistics, and to write the interval as the closed interval \(\cintf{X}\). The macros \cilof{x} and \ciupf{x} print \(\cilof{x}\) and \(\ciupf{x}\), and \cintf{x} prints \(\cintf{x}\).

We chose this form for two reasons.

It matches Casella and Berger, who define an interval estimator as the random interval \([L(\vX), U(\vX)]\) (Casella and Berger, Statistical Inference, 2nd ed., 2002, Definition 9.1.1).
It packages the two random endpoints into a single random interval, rather than placing the parameter between two random bounds. In \(\Pr\paren{\theta \in \cintf{X}}\), the event \(\theta \in \cintf{X}\) is then an event about the random data \(X\), and the probability is over \(X\) with \(\theta\) held fixed.

The macro \ci prints the Wald form \(\ci\) of one such interval, with the \(\pm\) sign in red, and is not a general notation for the interval.