Notation
This page follows the notation used throughout the Morrison Lab’s course materials, summarized here.
- Random variables are denoted with uppercase letters (\(X\), \(Y\), \(Z\)), and their realized (observed) values with the matching lowercase letters (\(x\), \(y\), \(z\)).
- Probability is denoted \(\Pr()\) or \(\mathbb{P}()\) for the probability of an event, or for a probability mass function (PMF), and \(\mathbb{p}()\) for a density.
- Expectation is denoted \({\mathbb{E}\mathopen{}\left[\cdot\right]\mathclose{}}\), with square brackets.
- Independence is denoted \(\perp\!\!\!\perp\) (read “\(X \perp\!\!\!\perp Y\)” as “\(X\) is independent of \(Y\)”).
- Complements of events are denoted \(\neg A\) (“not \(A\)”).
- We write \(\stackrel{\text{def}}{=}\) for an equality that holds by definition, to distinguish it from an equality that follows from other facts.
- Other sources often use other symbols for these. The notation choices page lists the alternatives; always check a new source’s own notation section before assuming ours.
- The full macro list, with more notational variants, is in
latex-macros.
1 Stochastic vs. probabilistic vs. random
Remark. The terms “stochastic”, “probabilistic”, and “random” are frequently used in statistics and probability theory, often interchangeably in everyday conversation, but they carry nuanced technical distinctions.
1.0.1 Key distinction: modeling approach vs. phenomena
Remark. As noted in Wikipedia:
Stochasticity and randomness are technically distinct concepts: the former refers to a modeling approach, while the latter describes phenomena; in everyday conversation these terms are often used interchangeably.
Definition 1 (Random) Something is random when it occurs by chance, without a deterministic pattern: its outcome cannot be predicted precisely, only probabilistically. It is the most general of the three terms, used to describe variables or occurrences rather than whole processes or modeling approaches.
Remark. The term “random” is sometimes used as shorthand for a uniform distribution (especially the discrete uniform distribution), but it can refer to any probability distribution.
Example 1 (A random variable) The result of a single die roll is random: it cannot be predicted with certainty, only described by the probability \(1/6\) for each face. We speak of a “random variable” and a “random event” for exactly this kind of single outcome.
Remark 1 (The word “stochastic”). The word “stochastic” comes from the Greek \(\sigma\tau\acute{o}\chi o\varsigma\) (stókhos), meaning “aim” or “guess” (see etymology). The term is almost always used for processes or systems evolving in time or space under uncertain rules, rather than for a single variable or event.
Example 2 (Stock price evolution) The sequence of a stock’s daily closing prices is a stochastic process: a random variable (the price) indexed by a set (the trading days).
Definition 3 (Probabilistic) A model, method, or line of reasoning is probabilistic when it explicitly involves probability theory: it assigns probabilities to events or outcomes, and focuses on quantifying and reasoning about uncertainty based on known or estimated distributions. While every stochastic model is probabilistic (since it uses probabilities), not every probabilistic model needs to describe a process evolving in time.
Example 3 (A linear regression model) A linear regression model with Gaussian errors, \(Y = \beta_{0} + \beta_{1} x + \varepsilon\) with \(\varepsilon\sim {\operatorname{N}\mathopen{}\left(0, \sigma^2\right)\mathclose{}}\), is probabilistic: for each value of the covariate \(x\), it assigns a probability distribution to the outcome \(Y\). It is not a stochastic process, because it describes the outcome at a given covariate value rather than a process evolving in time.
Remark. The model is probabilistic because of its error distribution, not because of the method used to fit it. The same model is probabilistic whether its parameters are estimated by maximum likelihood or by Bayesian inference. Bayesian inference additionally assigns a probability distribution to the parameters themselves, which makes the inference method probabilistic too.
1.0.2 Summary of usage
Remark. While some sources treat “stochastic” and “random” as practically synonymous, a common convention is to use “random” for variables and events, and “stochastic” for processes, especially to highlight temporal or spatial structure in the modeling.