Estimation defines the accuracy of an estimator at one sample size, through its bias, variance and population mean squared error. This page asks two further questions: what happens to an estimator as the sample size grows, and how small can the variance of an unbiased estimator be?
Definition 1 (Consistent estimator) Let \(\hat{\theta}_n\) be an estimator of \(\theta\) computed from \(n\) observations. The estimator is consistent if, for every distance \(\varepsilon > 0\), the probability that it lands at least \(\varepsilon\) away from \(\theta\) goes to zero as the sample size grows:
In words, \(\hat{\theta}_n\)converges in probability to \(\theta\), which we write \(\hat{\theta}_n \overset{\scriptstyle p}{\longrightarrow}\theta\).
Source: Casella and Berger [1], Def. 10.1.1; Wikipedia contributors [2].
RemarkRemark 1 (Consistency describes a sequence of estimators)
Remark 1 (Consistency describes a sequence of estimators). An estimator such as the sample mean is a formula that applies to any sample size, so \(\hat{\theta}_1, \hat{\theta}_2, \ldots\) is a sequence of estimators, one for each \(n\). Unbiasedness and pMSE describe an estimator at one fixed sample size; consistency describes what happens to the whole sequence as \(n\) grows. An estimator can be consistent and biased at every \(n\), or unbiased at every \(n\) and not consistent (the single-observation estimator \(X_1\) of a mean is the standard example).
ExrExercise 1 (Bounding the chance of a large estimation error)
Exercise 1 (Bounding the chance of a large estimation error) Let \(\hat{\theta}\) be an estimator of \(\theta\) with finite pMSE, and let \(\varepsilon > 0\). Use Markov’s inequality to show that \({\Pr\mathopen{}\left(\mathopen{}\left|\hat{\theta}- \theta\right|\mathclose{} \ge \varepsilon\right)\mathclose{}} \le {\operatorname{pMSE}\mathopen{}\left(\hat{\theta}\right)\mathclose{}} / \varepsilon^2\).
Solution 1. Both sides of \(\mathopen{}\left|\hat{\theta}- \theta\right|\mathclose{} \ge \varepsilon\) are nonnegative, so squaring them keeps the inequality. The random variable \({\mathopen{}\left(\hat{\theta}- \theta\right)\mathclose{}}^{2}\) is nonnegative, so Markov’s inequality applies to it with \(a = \varepsilon^2\):
\[
\begin{aligned}
{\Pr\mathopen{}\left(\mathopen{}\left|\hat{\theta}- \theta\right|\mathclose{} \ge \varepsilon\right)\mathclose{}}
&= {\Pr\mathopen{}\left({\mathopen{}\left(\hat{\theta}- \theta\right)\mathclose{}}^{2} \ge \varepsilon^2\right)\mathclose{}}
&& \text{(square both nonnegative sides)}\\
&\le \frac{{\mathbb{E}\mathopen{}\left[{\mathopen{}\left(\hat{\theta}- \theta\right)\mathclose{}}^{2}\right]\mathclose{}}}{\varepsilon^2}
&& \text{(Markov's inequality, $a = \varepsilon^2 > 0$)}\\
&= \frac{{\mathbb{E}\mathopen{}\left[{\mathopen{}\left({\varepsilon\mathopen{}\left(\hat{\theta}\right)\mathclose{}}\right)\mathclose{}}^{2}\right]\mathclose{}}}{\varepsilon^2}
&& \text{(definition of the estimation error)}\\
&= \frac{{\operatorname{pMSE}\mathopen{}\left(\hat{\theta}\right)\mathclose{}}}{\varepsilon^2}
&& \text{(definition of the mean squared error)}
\end{aligned}
\]
Lemma 1 (Chance of a large estimation error) If \(\hat{\theta}\) is an estimator of \(\theta\) with finite pMSE, then for every \(\varepsilon > 0\) the probability of an estimation error of at least \(\varepsilon\) is at most the pMSE divided by \(\varepsilon^2\):
LemTheorem 1 (Vanishing population mean squared error implies consistency)
Theorem 1 (Vanishing population mean squared error implies consistency) If the pMSE of \(\hat{\theta}_n\) goes to zero as \(n\) grows, then \(\hat{\theta}_n\) is a consistent estimator of \(\theta\):
Proof. Let \(\varepsilon > 0\). By Lemma 1, \(0 \le {\Pr\mathopen{}\left(\mathopen{}\left|\hat{\theta}_n - \theta\right|\mathclose{} \ge \varepsilon\right)\mathclose{}} \le {\operatorname{pMSE}\mathopen{}\left(\hat{\theta}_n\right)\mathclose{}} / \varepsilon^2\) for every \(n\). The upper bound goes to zero as \(n \to \infty\), because \(\varepsilon\) is fixed and \({\operatorname{pMSE}\mathopen{}\left(\hat{\theta}_n\right)\mathclose{}} \to 0\), so the probability, squeezed between zero and the bound, goes to zero. This holds for every \(\varepsilon > 0\), which is Equation 1.
Corollary 1 (The sample mean is consistent) Let \(X_1, \ldots, X_n\) be independent with common mean \(\mu\) and common finite variance \(\sigma^2\). Then the sample mean is a consistent estimator of \(\mu\): \(\bar X_n \overset{\scriptstyle p}{\longrightarrow}\mu\).
ProofProof
Proof. By the example on two estimators of a mean, \({\operatorname{pMSE}\mathopen{}\left(\bar X_n\right)\mathclose{}} = \sigma^2/ n\), which goes to zero as \(n \to \infty\). Theorem 1 then gives \(\bar X_n \overset{\scriptstyle p}{\longrightarrow}\mu\). This is the weak law of large numbers, proved in pds directly from Chebyshev’s inequality.
ExmExample 1 (Consistency of the sample mean for birth weights)
Example 1 (Consistency of the sample mean for birth weights) The birthwt data record the birth weight (grams) of 189 babies from a study at Baystate Medical Center in Springfield, Massachusetts [3]; the data ship with the R package MASS (MASS::birthwt), and this site has a copy at data/birthwt.csv.
To see Corollary 1 at work, treat these babies as a finite population and let the estimand \(\mu\) be their population mean birth weight. We draw \(n\) babies at random with replacement, so the draws are independent with the population’s mean and variance, and we ask how often the sample mean misses \(\mu\) by at least \(\varepsilon = 100\) grams. For each \(n\) we repeat the sampling 2000 times and report the fraction of repeats that miss, with a 95% confidence interval for that probability (the Wilson score interval with continuity correction, the default of prop.test()), next to the bound \(\sigma^2/ (n \varepsilon^2)\) that Lemma 1 gives, capped at 1 because a probability cannot exceed 1, using \({\operatorname{pMSE}\mathopen{}\left(\bar X_n\right)\mathclose{}} = \sigma^2/ n\) from the example on two estimators of a mean.
Show R code
birthwt<-read.csv("data/birthwt.csv")bwt<-birthwt$bwt# The population: the 189 observed birth weights.mu<-mean(bwt)# Population variance (divisor N, not N - 1), because the draws below# come from exactly this finite population.sigma_sq<-mean((bwt-mu)^2)eps<-100n_reps<-2000sample_sizes<-c(5, 10, 20, 40, 80, 160, 320)set.seed(203)miss_counts<-vapply(sample_sizes,function(n){sum(replicate(n_reps,abs(mean(sample(bwt, n, replace =TRUE))-mu)>=eps))},numeric(1))miss_ci<-lapply(miss_counts, prop.test, n =n_reps)consistency_sim<-data.frame( n =sample_sizes, miss_prob =miss_counts/n_reps, lower =vapply(miss_ci, function(ci)ci$conf.int[1], numeric(1)), upper =vapply(miss_ci, function(ci)ci$conf.int[2], numeric(1)), bound =pmin(1, sigma_sq/(sample_sizes*eps^2)))consistency_sim
Show R code
p<-ggplot(consistency_sim, aes(x =n))+geom_ribbon(aes(ymin =lower, ymax =upper), alpha =0.25)+geom_line(aes(y =miss_prob, linetype ="simulated"))+geom_point(aes(y =miss_prob))+geom_line(aes(y =bound, linetype ="bound"))+scale_x_log10(breaks =sample_sizes)+scale_linetype_manual( values =c(simulated ="solid", bound ="dashed"), name =NULL)+labs( x ="sample size n", y ="probability of a miss")if(knitr::is_html_output())plotly::ggplotly(p)elsep
Figure 1: The chance that the sample mean of \(n\) birth weights misses the population mean by 100 g or more, by simulation (2000 repeats per \(n\), with 95% confidence bands) and by the bound from Lemma 1
The simulated probability shrinks toward zero as \(n\) grows, as consistency says it must, and it lies below the bound at every sample size here, which is loose for small \(n\) but follows the same \(1/n\) decay.
Definition 2 (Relative efficiency) Let \(\hat{\theta}_a\) and \(\hat{\theta}_b\) be two estimators of the same estimand \(\theta\). The relative efficiency of \(\hat{\theta}_a\) with respect to \(\hat{\theta}_b\) is the ratio of their pMSEs, the reference estimator’s, that of \(\hat{\theta}_b\), in the numerator:
Remark 2 (Source: Relative efficiency). Casella and Berger [1] define the asymptotic version, through variances, in Def. 10.1.16 (p. 476); this finite-sample form uses the mean squared error of their Def. 7.3.1 (p. 330).
Remark 3 (Reading the relative efficiency). A relative efficiency above 1 means \(\hat{\theta}_a\) is more accurate than \(\hat{\theta}_b\), and below 1 means it is less accurate. For two unbiased estimators the pMSEs are the variances (pMSE equals bias squared plus variance, with zero bias), so the relative efficiency is the ratio of the variances.
ExmExample 2 (Relative efficiency of two estimators of a mean)
Example 2 (Relative efficiency of two estimators of a mean) In the example on two estimators of a mean the sample mean has \({\operatorname{pMSE}\mathopen{}\left(\bar X\right)\mathclose{}} = \sigma^2/ n\) and the single-observation estimator has \({\operatorname{pMSE}\mathopen{}\left(X_1\right)\mathclose{}} = \sigma^2\). The relative efficiency of the sample mean with respect to \(X_1\) is
\[
\begin{aligned}
\frac{{\operatorname{pMSE}\mathopen{}\left(X_1\right)\mathclose{}}}{{\operatorname{pMSE}\mathopen{}\left(\bar X\right)\mathclose{}}}
&= \frac{\sigma^2}{\sigma^2/ n}
&& \text{(substitute the two mean squared errors)}\\
&= \sigma^2\cdot \frac{n}{\sigma^2}
&& \text{(divide by a fraction: multiply by its reciprocal)}\\
&= n \cdot \frac{\sigma^2}{\sigma^2}
&& \text{(reorder the factors)}\\
&= n \cdot 1
&& \text{($a / a = 1$)}\\
&= n
&& \text{($b \cdot 1 = b$)}
\end{aligned}
\]
so with \(n = 50\) students, as in the student-height example, the sample mean is 50 times as efficient as \(X_1\).
ReadingRead
Casella and Berger [1, Sec. 10.1], pp. 467–480 covers point estimation, including consistency, efficiency and bootstrap standard errors.
3 The Cramér–Rao lower bound
ExrExercise 2 (Covariance of an unbiased estimator with the score)
Exercise 2 (Covariance of an unbiased estimator with the score) Let \(\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{X}}\vphantom{X}}\) have density \(\operatorname{p}(\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{x}}\vphantom{x}}\mid \theta)\) for a scalar parameter \(\theta\), under the assumptions of the exercise on the mean of the score, and let \(\hat{\theta}= \hat{\theta}(\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{X}}\vphantom{X}})\) be an unbiased estimator of \(\theta\) with finite variance. Show that the covariance of \(\hat{\theta}\) with the score\(\ell'(\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{X}}\vphantom{X}}\mid \theta)\) equals 1. Assume the derivative of \({\mathbb{E}\mathopen{}\left[\hat{\theta}\right]\mathclose{}}\) with respect to \(\theta\) can be taken inside the integral.
\[
\begin{aligned}
{\operatorname{Cov}\mathopen{}\left(\hat{\theta}, \ell'\right)\mathclose{}}
&= {\mathbb{E}\mathopen{}\left[\hat{\theta}\ell'\right]\mathclose{}} - {\mathbb{E}\mathopen{}\left[\hat{\theta}\right]\mathclose{}}\,{\mathbb{E}\mathopen{}\left[\ell'\right]\mathclose{}}
&& \text{(definition of covariance)}\\
&= {\mathbb{E}\mathopen{}\left[\hat{\theta}\ell'\right]\mathclose{}} - {\mathbb{E}\mathopen{}\left[\hat{\theta}\right]\mathclose{}} \cdot 0
&& \text{(the score has mean zero)}\\
&= {\mathbb{E}\mathopen{}\left[\hat{\theta}\ell'\right]\mathclose{}} - 0
&& \text{($b \cdot 0 = 0$)}\\
&= {\mathbb{E}\mathopen{}\left[\hat{\theta}\ell'\right]\mathclose{}}
&& \text{($a - 0 = a$)}\\
&= \int \hat{\theta}(\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{x}}\vphantom{x}}) \, \ell'(\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{x}}\vphantom{x}}\mid \theta) \, \operatorname{p}(\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{x}}\vphantom{x}}\mid \theta) \, d\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{x}}\vphantom{x}}
&& \text{(definition of expectation)}\\
&= \int \hat{\theta}(\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{x}}\vphantom{x}}) \mathopen{}\left(\frac{\partial}{\partial \theta} \log \operatorname{p}(\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{x}}\vphantom{x}}\mid \theta)\right)\mathclose{} \operatorname{p}(\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{x}}\vphantom{x}}\mid \theta) \, d\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{x}}\vphantom{x}}
&& \text{(definition of the score)}\\
&= \int \hat{\theta}(\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{x}}\vphantom{x}}) \, \frac{\partial}{\partial \theta} \operatorname{p}(\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{x}}\vphantom{x}}\mid \theta) \, d\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{x}}\vphantom{x}}
&& \text{(derivative of a log density)}\\
&= \frac{\partial}{\partial \theta} \int \hat{\theta}(\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{x}}\vphantom{x}}) \, \operatorname{p}(\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{x}}\vphantom{x}}\mid \theta) \, d\mathord{\smash[b]{\underset{\raise.3ex\hbox{$\scriptstyle\sim$}}{x}}\vphantom{x}}
&& \text{(exchange derivative and integral)}\\
&= \frac{\partial}{\partial \theta} {\mathbb{E}\mathopen{}\left[\hat{\theta}\right]\mathclose{}}
&& \text{(definition of expectation)}\\
&= \frac{\partial}{\partial \theta} \theta
&& \text{($\hat{\theta}$ is unbiased)}\\
&= 1
&& \text{(derivative of $\theta$ with respect to $\theta$)}
\end{aligned}
\]
LemLemma 2 (Covariance of an unbiased estimator with the score)
Lemma 2 (Covariance of an unbiased estimator with the score) Under the assumptions of Exercise 2, an unbiased estimator \(\hat{\theta}\) of \(\theta\) has covariance 1 with the score:
Theorem 2 (Cramér–Rao lower bound) Under the assumptions of Exercise 2, suppose the expected information satisfies \(0 < \mathcal{I}(\theta) < \infty\). Then every unbiased estimator \(\hat{\theta}\) of \(\theta\) has variance at least the reciprocal of the expected information:
Proof. The score has variance \(\mathcal{I}(\theta) > 0\) by the theorem on the mean and variance of the score, and \({\operatorname{Cov}\mathopen{}\left(\hat{\theta}, \ell'\right)\mathclose{}} = 1\) by Lemma 2. A constant has covariance zero with every random variable, so \({\operatorname{Var}\mathopen{}\left(\hat{\theta}\right)\mathclose{}} > 0\). The correlation of \(\hat{\theta}\) and the score is therefore defined, and by its bounds its square is at most 1:
\[
\begin{aligned}
1
&\ge {\mathopen{}\left({\operatorname{Cor}\mathopen{}\left(\hat{\theta}, \ell'\right)\mathclose{}}\right)\mathclose{}}^{2}
&& \text{(correlation lies in $[-1, 1]$)}\\
&= \frac{{\mathopen{}\left({\operatorname{Cov}\mathopen{}\left(\hat{\theta}, \ell'\right)\mathclose{}}\right)\mathclose{}}^{2}}{{\operatorname{Var}\mathopen{}\left(\hat{\theta}\right)\mathclose{}}\,{\operatorname{Var}\mathopen{}\left(\ell'\right)\mathclose{}}}
&& \text{(definition of correlation, squared)}\\
&= \frac{1^2}{{\operatorname{Var}\mathopen{}\left(\hat{\theta}\right)\mathclose{}}\,\mathcal{I}(\theta)}
&& \text{(covariance is 1; the score has variance $\mathcal{I}(\theta)$)}\\
&= \frac{1}{{\operatorname{Var}\mathopen{}\left(\hat{\theta}\right)\mathclose{}}\,\mathcal{I}(\theta)}
&& \text{($1^2 = 1$)}
\end{aligned}
\]
Multiplying both sides by \({\operatorname{Var}\mathopen{}\left(\hat{\theta}\right)\mathclose{}}\) and dividing by \(\mathcal{I}(\theta)\), both positive, gives Equation 5.
RemarkRemark 4 (An efficient estimator has the smallest variance)
Remark 4 (An efficient estimator has the smallest variance). By the Cramér–Rao lower bound, no unbiased estimator that satisfies its assumptions has a smaller variance than an efficient one. Among such estimators, an efficient estimator therefore also has the smallest pMSE (pMSE equals bias squared plus variance, with zero bias).
ExmExample 3 (The sample mean is efficient for Poisson data)
Example 3 (The sample mean is efficient for Poisson data) For \(X_1, \ldots, X_n \ \sim_{\operatorname{iid}}\ \operatorname{Pois}({\lambda})\), the sample mean \(\bar X\) is unbiased for \({\lambda}\) (the bias example), and, because \({\operatorname{Var}\mathopen{}\left(X_i\right)\mathclose{}} = {\lambda}\) for a Poisson variable, \({\operatorname{Var}\mathopen{}\left(\bar X\right)\mathclose{}} = {\lambda}/ n\) (the example on two estimators of a population mean). The expected information is \(\mathcal{I}({\lambda}) = n / {\lambda}\) (the Poisson information example), so
\[
\begin{aligned}
\frac{1}{\mathcal{I}({\lambda})}
&= \frac{1}{n / {\lambda}}
&& \text{(substitute the expected information)}\\
&= \frac{{\lambda}}{n}
&& \text{(reciprocal of a fraction)}\\
&= {\operatorname{Var}\mathopen{}\left(\bar X\right)\mathclose{}}
&& \text{(variance of the sample mean)}
\end{aligned}
\]
The sample mean attains the Cramér–Rao bound, so it is an efficient estimator of \({\lambda}\). It is also the maximum likelihood estimator (the Poisson MLE example).
RemarkRemark 5 (What the bound does and does not say)
Remark 5 (What the bound does and does not say). The bound holds only for unbiased estimators, and only when the regularity conditions of the exercise on the mean of the score hold. When the set of possible data values depends on the parameter, as for the uniform distribution on \([0, \theta]\), the conditions fail and the bound need not hold (compare the uniform example).
A biased estimator can have a variance below the bound, because the bound relies on the covariance of an estimator and the score, which assumes the estimator is unbiased. Its pMSE adds the squared bias to the variance, so trading a little bias for a large drop in variance can also lower the pMSE.
The information from \(n\) independent observations is \(n\) times the information from one, so for a sample of size \(n\) the bound shrinks like \(1/n\). That is the rate at which the variance of an efficient estimator falls, and the maximum likelihood estimator attains it asymptotically.
ReadingRead
Hogg et al. [8, Sec. 6.6], pp. 285–289 covers asymptotic distributions of maximum likelihood estimators.
Casella and Berger [1, Sec. 7.3.2], pp. 334–341 covers best unbiased estimators and the Cramér–Rao bound.
Rice [9, Sec. 8.5.2], pp. 274–278 covers large-sample theory for maximum likelihood estimators.
Rice [9, Sec. 8.7], pp. 298–304 covers efficiency and the Cramér–Rao lower bound.
C. R. Rao, “Information and the accuracy attainable in the estimation of statistical parameters,”Bulletin of the Calcutta Mathematical Society, vol. 37, pp. 81–91, 1945.
[6]
H. Cramér, Mathematical methods of statistics. Princeton, NJ: Princeton University Press, 1946. doi: 10.1515/9781400883868.
---title: "Properties of Estimators"format: html: default revealjs: output-file: estimator-properties-slides.html pdf: output-file: estimator-properties-handout.pdf docx: output-file: estimator-properties-handout.docxresources: - data/birthwt.csv---{{< include sds/_subfiles/shared-config.qmd >}}[Estimation](estimation.qmd) defines the accuracy of an estimatorat one sample size, through its bias, variance and population mean squared error.This page asks two further questions:what happens to an estimator as the sample size grows,and how small can the variance of an unbiased estimator be?## Consistency {#sec-consistency}{{< include sds/_subfiles/estimator-properties/_def-consistent.qmd >}}{{< slidebreak >}}{{< include sds/_subfiles/estimator-properties/_rem-consistent-sequence.qmd >}}{{< slidebreak >}}{{< include sds/_subfiles/estimator-properties/_exr-estimation-error-bound.qmd >}}{{< slidebreak >}}{{< include sds/_subfiles/estimator-properties/_lem-estimation-error-bound.qmd >}}{{< slidebreak >}}{{< include sds/_subfiles/estimator-properties/_thm-mse-consistent.qmd >}}{{< slidebreak >}}{{< include sds/_subfiles/estimator-properties/_cor-sample-mean-consistent.qmd >}}{{< slidebreak >}}{{< include sds/_subfiles/estimator-properties/_exm-consistency-birthwt.qmd >}}## Efficiency {#sec-efficiency}{{< include sds/_subfiles/estimator-properties/_def-relative-efficiency.qmd >}}{{< slidebreak >}}{{< include sds/_subfiles/estimator-properties/_exm-relative-efficiency-mean.qmd >}}::: {.reading}@CaseBerg01 [Sec. 10.1, pp. 467--480] covers point estimation, including consistency, efficiency and bootstrap standard errors.:::## The Cramér--Rao lower bound {#sec-cramer-rao}{{< include sds/_subfiles/estimator-properties/_exr-cov-estimator-score.qmd >}}{{< slidebreak >}}{{< include sds/_subfiles/estimator-properties/_lem-cov-estimator-score.qmd >}}{{< slidebreak >}}{{< include sds/_subfiles/estimator-properties/_thm-cramer-rao.qmd >}}{{< slidebreak >}}{{< include sds/_subfiles/estimator-properties/_def-efficient.qmd >}}{{< slidebreak >}}{{< include sds/_subfiles/estimator-properties/_exm-poisson-efficient.qmd >}}{{< slidebreak >}}{{< include sds/_subfiles/estimator-properties/_rem-cramer-rao-scope.qmd >}}::: {.reading}- @hoggtanis2015 [Sec. 6.6, pp. 285--289] covers asymptotic distributions of maximum likelihood estimators.- @CaseBerg01 [Sec. 7.3.2, pp. 334--341] covers best unbiased estimators and the Cramér--Rao bound.- @rice2007mathematical [Sec. 8.5.2, pp. 274--278] covers large-sample theory for maximum likelihood estimators.- @rice2007mathematical [Sec. 8.7, pp. 298--304] covers efficiency and the Cramér--Rao lower bound.:::## References {.unnumbered}::: {#refs}:::