Measures

Remark. A measure assigns a size to each set in a collection of sets: how many elements it has, how long it is, or how likely it is. This page builds measures from \(\sigma\)-algebras and additivity, using the sets and functions page’s definitions. The Morrison Lab’s probability notes define a probability measure as a measure that gives the whole space size 1 (probability measure).

1 \(\sigma\)-algebras

Definition 1 (\(\sigma\)-algebra) A \(\sigma\)-algebra on a set \(S\) is a collection \(\mathscr{S}\) of subsets of \(S\) that satisfies:

• \(\mathscr{S}\) contains \(S\) itself.
• For each set \(A\) in \(\mathscr{S}\), \(\mathscr{S}\) contains its complement \(S \setminus A\).
• For each sequence \(A_1, A_2, \ldots\) of sets in \(\mathscr{S}\), \(\mathscr{S}\) contains their union \(\bigcup_{i=1}^{\infty} A_i\).

Remark. Other sources call it a \(\sigma\)-field (see Wikipedia: \(\sigma\)-algebra).

Example 1 (\(\sigma\)-algebras for die rolls) For the die rolls \(D = \mathopen{}\left\{1, 2, 3, 4, 5, 6\right\}\mathclose{}\) (sets of die rolls), the collection of all subsets of \(D\) is a \(\sigma\)-algebra on \(D\), since complements and unions of subsets of \(D\) are again subsets of \(D\). So is the smaller collection

\[\mathopen{}\left\{\emptyset, \mathopen{}\left\{2, 4, 6\right\}\mathclose{}, \mathopen{}\left\{1, 3, 5\right\}\mathclose{}, D\right\}\mathclose{}\]

which contains \(D\), the complement of each of its sets, and every union of its sets: for example, \(\mathopen{}\left\{2, 4, 6\right\}\mathclose{} \cup \mathopen{}\left\{1, 3, 5\right\}\mathclose{} = D\).

Theorem 1 (Closure properties of a \(\sigma\)-algebra) If \(\mathscr{S}\) is a \(\sigma\)-algebra on a set \(S\), then \(\mathscr{S}\) contains:

• the empty set \(\emptyset\);
• the union \(A_1 \cup \cdots \cup A_n\) of any finitely many sets \(A_1, \ldots, A_n\) in \(\mathscr{S}\);
• the intersection \(\bigcap_{i=1}^{\infty} A_i\) of any sequence \(A_1, A_2, \ldots\) of sets in \(\mathscr{S}\).

ProofProof

Proof. Empty set. \(\mathscr{S}\) contains \(S\), so it contains the complement of \(S\), which is \(S \setminus S = \emptyset\).

Finite unions. Extend \(A_1, \ldots, A_n\) to a sequence by setting \(A_{n+1} = A_{n+2} = \cdots = \emptyset\); each of these sets is in \(\mathscr{S}\), by the first part. Then:

\[ \begin{aligned} A_1 \cup \cdots \cup A_n &= A_1 \cup \cdots \cup A_n \cup \emptyset \cup \emptyset \cup \cdots && \text{(a union with } \emptyset \text{ adds no elements)} \\ &= \bigcup_{i=1}^{\infty} A_i && \text{(} A_i = \emptyset \text{ for } i > n \text{)} \end{aligned} \]

and \(\mathscr{S}\) contains \(\bigcup_{i=1}^{\infty} A_i\) by the union rule of Definition 1.

Countable intersections. An element of \(S\) is in every \(A_i\) exactly when it is in none of the complements \(S \setminus A_i\), so:

\[ \begin{aligned} \bigcap_{i=1}^{\infty} A_i &= S \setminus \bigcup_{i=1}^{\infty} (S \setminus A_i) && \text{(De Morgan's law)} \end{aligned} \]

Each \(S \setminus A_i\) is in \(\mathscr{S}\) by the complement rule, so their union is in \(\mathscr{S}\) by the union rule, and the complement of that union is in \(\mathscr{S}\) by the complement rule again.

Theorem 2 (All subsets form a \(\sigma\)-algebra) For any set \(S\), the collection of all subsets of \(S\) is a \(\sigma\)-algebra on \(S\).

ProofProof

Proof. Each condition of Definition 1 holds:

• \(S\) is a subset of itself.
• For each subset \(A\) of \(S\), the complement \(S \setminus A\) contains only elements of \(S\), so it is a subset of \(S\).
• For each sequence \(A_1, A_2, \ldots\) of subsets of \(S\), every element of \(\bigcup_{i=1}^{\infty} A_i\) is in some \(A_i\) and so in \(S\); the union is a subset of \(S\).

2 Pairwise disjoint sets

Definition 2 (Pairwise disjoint sets) Finitely or countably many sets \(A_1, A_2, \ldots\) are pairwise disjoint (also called disjoint or mutually disjoint) when no two of them share an element:

\[A_i \cap A_j = \emptyset \quad \text{for all } i \neq j\]

Remark. In probability, pairwise disjoint events are usually called mutually exclusive.

Example 2 (Low and high die rolls) For the die rolls \(D = \mathopen{}\left\{1, 2, 3, 4, 5, 6\right\}\mathclose{}\) (sets of die rolls), the sets \(\mathopen{}\left\{1, 2\right\}\mathclose{}\) and \(\mathopen{}\left\{5, 6\right\}\mathclose{}\) are pairwise disjoint: no roll is in both. The sets \(\mathopen{}\left\{2, 4, 6\right\}\mathclose{}\) and \(\mathopen{}\left\{1, 2\right\}\mathclose{}\) are not, because \(2\) is in both.

3 Additivity

Definition 3 (Finite additivity) Let \(\mathscr{S}\) be a \(\sigma\)-algebra on a set \(S\). A function \(\mu : \mathscr{S} \to [0, \infty]\), with values in the extended non-negative reals, is finitely additive if, for every finite collection of pairwise disjoint sets \(A_1, \ldots, A_n\) in \(\mathscr{S}\), the value of their union is the sum of their values:

\[\mu(A_1 \cup \cdots \cup A_n) = \sum_{i=1}^{n} \mu(A_i)\]

Example 3 (Counting elements is finitely additive) For the die rolls \(D = \mathopen{}\left\{1, 2, 3, 4, 5, 6\right\}\mathclose{}\) (sets of die rolls), let \(\mu(A) \stackrel{\text{def}}{=}\mathopen{}\left|A\right|\mathclose{}\), the number of elements of \(A\), for each set \(A\) in the \(\sigma\)-algebra of all subsets of \(D\) (Example 1). The sets \(\mathopen{}\left\{1, 2\right\}\mathclose{}\) and \(\mathopen{}\left\{5, 6\right\}\mathclose{}\) are pairwise disjoint (Example 2), and:

\[ \begin{aligned} \mu(\mathopen{}\left\{1, 2\right\}\mathclose{} \cup \mathopen{}\left\{5, 6\right\}\mathclose{}) &= \mu(\mathopen{}\left\{1, 2, 5, 6\right\}\mathclose{}) && \text{(take the union)} \\ &= 4 && \text{(count the elements)} \\ &= 2 + 2 && \text{(write 4 as a sum)} \\ &= \mu(\mathopen{}\left\{1, 2\right\}\mathclose{}) + \mu(\mathopen{}\left\{5, 6\right\}\mathclose{}) && \text{(count each set's elements)} \end{aligned} \]

The same holds for any pairwise disjoint sets, because the sizes of disjoint sets add, so \(\mu\) is finitely additive.

Definition 4 (Countable additivity) Let \(\mathscr{S}\) be a \(\sigma\)-algebra on a set \(S\). A function \(\mu : \mathscr{S} \to [0, \infty]\) is countably additive (also called \(\sigma\)-additive) if, for every sequence of pairwise disjoint sets \(A_1, A_2, \ldots\) in \(\mathscr{S}\), the value of their union is the sum of their values:

\[\mu\!\left(\bigcup_{i=1}^{\infty} A_i\right) = \sum_{i=1}^{\infty} \mu(A_i)\]

Example 4 (Counting elements is countably additive) For the counting function \(\mu(A) \stackrel{\text{def}}{=}\mathopen{}\left|A\right|\mathclose{}\) of Example 3, take any sequence of pairwise disjoint sets \(A_1, A_2, \ldots\). No two of them share an element, and \(D\) has only six elements, so at most six of the \(A_i\) contain any elements; the rest are \(\emptyset\), with \(\mu(\emptyset) = 0\). The infinite sum therefore has at most six nonzero terms, and finite additivity (Example 3) shows that those terms add up to \(\mu\) of the union. So \(\mu\) is countably additive.

Lemma 1 (Sums of non-negative terms) Let \(a_1, a_2, \ldots\) be values in \([0, \infty]\), with partial sums \(s_n \stackrel{\text{def}}{=}\sum_{i=1}^{n} a_i\), where \(x + \infty = \infty\) for every \(x\) in \([0, \infty]\). Then the partial sums are non-decreasing, \(s_1 \le s_2 \le \cdots\), so their limit, the sum \(\sum_{i=1}^{\infty} a_i = \lim_{n \to \infty} s_n\), always exists, and it is either a finite number or \(\infty\).

ProofProof

Proof. For each \(n\):

\[ \begin{aligned} s_{n+1} &= s_n + a_{n+1} && \text{(definition of } s_{n+1} \text{)} \\ &\ge s_n && \text{(} a_{n+1} \ge 0 \text{)} \end{aligned} \]

If some partial sum \(s_N\) is \(\infty\), then \(s_n = \infty\) for every \(n \ge N\), so the limit is \(\infty\). Otherwise, the partial sums form a non-decreasing sequence of real numbers. If that sequence is bounded above, it converges to a finite number (see Wikipedia: Monotone convergence theorem). If it is not bounded above, then for every number \(M\) some \(s_N\) exceeds \(M\), and so does every later \(s_n \ge s_N\); the limit is \(\infty\).

Remark. By Lemma 1, the right-hand side of the equation in Definition 4 always has a value.

Lemma 2 (Countable additivity and the empty set) If \(\mu\) is a countably additive function on a \(\sigma\)-algebra \(\mathscr{S}\), then \(\mu(\emptyset) = 0\) or \(\mu(\emptyset) = \infty\).

ProofProof

Proof. The set \(\emptyset = S \setminus S\) is in \(\mathscr{S}\), as the complement of \(S\), and the sequence \(\emptyset, \emptyset, \ldots\) is pairwise disjoint with union \(\emptyset\), so:

\[ \begin{aligned} \mu(\emptyset) &= \mu\!\left(\bigcup_{i=1}^{\infty} \emptyset\right) && \text{(the union of copies of } \emptyset \text{ is } \emptyset \text{)} \\ &= \sum_{i=1}^{\infty} \mu(\emptyset) && \text{(countable additivity)} \end{aligned} \]

If \(\mu(\emptyset) = c\) for a finite \(c > 0\), the right-hand side is \(c + c + \cdots = \infty \neq c\), a contradiction. So \(\mu(\emptyset)\) is 0 or \(\infty\).

Example 5 (A countably additive function with \(\mu(\emptyset) = 0\)) The counting function \(\mu(A) \stackrel{\text{def}}{=}\mathopen{}\left|A\right|\mathclose{}\) of Example 4 is countably additive, and:

\[ \begin{aligned} \mu(\emptyset) &= \mathopen{}\left|\emptyset\right|\mathclose{} && \text{(definition of } \mu \text{)} \\ &= 0 && \text{(} \emptyset \text{ has no elements)} \end{aligned} \]

Example 6 (A countably additive function with \(\mu(\emptyset) = \infty\)) For the die rolls \(D = \mathopen{}\left\{1, 2, 3, 4, 5, 6\right\}\mathclose{}\) (sets of die rolls), let \(\mu(A) \stackrel{\text{def}}{=}\infty\) for every set \(A\) in the \(\sigma\)-algebra of all subsets of \(D\) (Example 1), including \(A = \emptyset\). For any sequence of pairwise disjoint sets \(A_1, A_2, \ldots\), the left-hand side of the countable additivity equation is:

\[ \begin{aligned} \mu\!\left(\bigcup_{i=1}^{\infty} A_i\right) &= \infty && \text{(definition of } \mu \text{)} \end{aligned} \]

and the right-hand side, the limit of its partial sums (Lemma 1), is:

\[ \begin{aligned} \sum_{i=1}^{\infty} \mu(A_i) &= \infty + \infty + \cdots && \text{(definition of } \mu \text{)} \\ &= \infty && \text{(every partial sum is } \infty \text{)} \end{aligned} \]

The two sides agree, so \(\mu\) is countably additive, and \(\mu(\emptyset) = \infty\).

Remark. Example 5 and Example 6 show that both values allowed by Lemma 2 occur. So \(\mu(\emptyset) = 0\) is an extra requirement, not a consequence of countable additivity.

Theorem 3 (Countable additivity implies finite additivity) If \(\mu\) is a countably additive function on a \(\sigma\)-algebra \(\mathscr{S}\), and \(\mu(\emptyset) = 0\), then \(\mu\) is finitely additive.

ProofProof

Proof. Let \(A_1, \ldots, A_n\) be pairwise disjoint sets in \(\mathscr{S}\), and extend them to a sequence by setting \(A_{n+1} = A_{n+2} = \cdots = \emptyset\). The extended sequence is still pairwise disjoint, since \(\emptyset\) shares no element with any set, and its union is \(A_1 \cup \cdots \cup A_n\). So:

\[ \begin{aligned} \mu(A_1 \cup \cdots \cup A_n) &= \mu\!\left(\bigcup_{i=1}^{\infty} A_i\right) && \text{(} A_i = \emptyset \text{ for } i > n \text{)} \\ &= \sum_{i=1}^{\infty} \mu(A_i) && \text{(countable additivity)} \\ &= \sum_{i=1}^{n} \mu(A_i) + \sum_{i=n+1}^{\infty} \mu(\emptyset) && \text{(} A_i = \emptyset \text{ for } i > n \text{)} \\ &= \sum_{i=1}^{n} \mu(A_i) && \text{(} \mu(\emptyset) = 0 \text{)} \end{aligned} \]

Example 7 (Finitely additive but not countably additive) Let \(S = \mathopen{}\left\{0, 1, 2, \ldots\right\}\mathclose{}\), with the \(\sigma\)-algebra of all subsets of \(S\) (Theorem 2), and define:

\[ \mu(A) \stackrel{\text{def}}{=}\begin{cases} 0 & \text{if } A \text{ is finite} \\ \infty & \text{if } A \text{ is infinite} \end{cases} \]

\(\mu\) is finitely additive. Let \(A_1, \ldots, A_n\) be pairwise disjoint subsets of \(S\). If every \(A_i\) is finite, then so is their union, and:

\[ \begin{aligned} \mu(A_1 \cup \cdots \cup A_n) &= 0 && \text{(a finite union of finite sets is finite)} \\ &= \sum_{i=1}^{n} 0 && \text{(a sum of zeros is 0)} \\ &= \sum_{i=1}^{n} \mu(A_i) && \text{(each } A_i \text{ is finite)} \end{aligned} \]

If some \(A_k\) is infinite, then the union, which contains \(A_k\), is infinite too, and:

\[ \begin{aligned} \mu(A_1 \cup \cdots \cup A_n) &= \infty && \text{(the union is infinite)} \\ &= \mu(A_k) + \sum_{i \neq k} \mu(A_i) && \text{(} \mu(A_k) = \infty \text{, and } \infty + x = \infty \text{)} \\ &= \sum_{i=1}^{n} \mu(A_i) && \text{(regroup the terms)} \end{aligned} \]

\(\mu\) is not countably additive. The single-element sets \(\mathopen{}\left\{0\right\}\mathclose{}, \mathopen{}\left\{1\right\}\mathclose{}, \mathopen{}\left\{2\right\}\mathclose{}, \ldots\) are pairwise disjoint, and their union is \(S\), which is infinite. So:

\[ \begin{aligned} \mu\!\left(\bigcup_{k=0}^{\infty} \mathopen{}\left\{k\right\}\mathclose{}\right) &= \mu(S) && \text{(the union is } S \text{)} \\ &= \infty && \text{(} S \text{ is infinite)} \end{aligned} \]

but:

\[ \begin{aligned} \sum_{k=0}^{\infty} \mu(\mathopen{}\left\{k\right\}\mathclose{}) &= 0 + 0 + \cdots && \text{(each } \mathopen{}\left\{k\right\}\mathclose{} \text{ is finite)} \\ &= 0 && \text{(every partial sum is 0)} \end{aligned} \]

Remark. Example 7 shows that the converse of Theorem 3 fails, even when \(\mu(\emptyset) = 0\): a finitely additive function need not be countably additive (see Wikipedia: Sigma-additive set function — An additive function which is not \(\sigma\)-additive).

4 Measures

Definition 5 (Measure) A measure on a set \(S\) with a \(\sigma\)-algebra \(\mathscr{S}\) is a function \(\mu : \mathscr{S} \to [0, \infty]\) that satisfies:

• \(\mu(\emptyset) = 0\).
• \(\mu\) is countably additive.

Example 8 (Counting elements is a measure) The counting function \(\mu(A) \stackrel{\text{def}}{=}\mathopen{}\left|A\right|\mathclose{}\) of Example 3, defined on the \(\sigma\)-algebra of all subsets of \(D\) (Example 1), takes values in \([0, \infty]\), is countably additive (Example 4), and gives \(\mu(\emptyset) = 0\), so it is a measure.

Corollary 1 (Measures are finitely additive) Every measure is finitely additive.

ProofProof

Proof. A measure \(\mu\) is countably additive and has \(\mu(\emptyset) = 0\) (Definition 5), so Theorem 3 applies to it.

Definition 6 (Counting measure) The counting measure on a set \(S\) is the measure on the \(\sigma\)-algebra of all subsets of \(S\) that assigns each finite set its number of elements, and each infinite set the value \(\infty\):

\[ \mu(A) \stackrel{\text{def}}{=}\begin{cases} \mathopen{}\left|A\right|\mathclose{} & \text{if } A \text{ is finite} \\ \infty & \text{if } A \text{ is infinite} \end{cases} \]

Example 9 (Counting measure on the non-negative integers) For the counting measure \(\mu\) on \(\mathopen{}\left\{0, 1, 2, \ldots\right\}\mathclose{}\), \(\mu(\mathopen{}\left\{0, 1, 2\right\}\mathclose{}) = 3\), and the set of even numbers has \(\mu(\mathopen{}\left\{0, 2, 4, \ldots\right\}\mathclose{}) = \infty\). The even numbers are the union of the pairwise disjoint sets \(\mathopen{}\left\{0\right\}\mathclose{}, \mathopen{}\left\{2\right\}\mathclose{}, \mathopen{}\left\{4\right\}\mathclose{}, \ldots\), and countable additivity agrees: \(\sum_{k=0}^{\infty} \mu(\mathopen{}\left\{2k\right\}\mathclose{}) = 1 + 1 + \cdots = \infty\).

Remark. A measure generalizes size: it can measure how many elements a set has, as in Definition 6, or how long a set of real numbers is, as Lebesgue measure does, assigning each interval \([a, b]\) its length \(b - a\). Both appear as reference measures in the Fubini–Tonelli theorem.