Central Limit Theorem

Overview

The central limit theorem describes the distribution of fluctuations in a large sum. It explains why normal approximations can arise from observations that are individually discrete or asymmetric. The approximation concerns a centered and rescaled sum, not a claim that the original observations become normal.

There are many central limit theorems. Here the scope is the classical iid theorem for real variables with finite positive variance. Dependence, unequal distributions, and infinite variance require separate hypotheses and sometimes different limits.

Prerequisites

For iid $X_1,X_2,\ldots$, set $S_n=\sum_{j=1}^n X_j$. Know expectation, variance, and convergence in distribution. Write $\Phi$ for the CDF of a standard normal variable.

Finite variance makes the natural scale of $S_n-n\mu$ equal to $\sigma\sqrt n$. Independence is what makes the variances add in this calculation.

Definition and conventions

Assume $\mu=E[X_1]$ and $0<\sigma^2=\operatorname{Var}(X_1)<\infty$. Then

Since $\Phi$ is continuous everywhere, this means $P(Z_n\le x)\to\Phi(x)$ for every real $x$. Equivalently, $\sqrt n(\overline X_n-\mu)/\sigma$ has this limit.

If $\sigma=0$, each $X_j=\mu$ almost surely; the displayed standardization is undefined. The theorem needs no third or fourth moment. Quantitative error bounds require additional conditions and do not follow from this statement alone.

Intuition

Centering removes the predictable linear growth, while division by $\sqrt n$ keeps the remaining variance fixed. Without this scaling, a sample average collapses toward its mean rather than retaining a nondegenerate limiting shape.

At the transform level, finite variance gives every centered, standardized summand the same leading quadratic behavior near the origin. Independence turns the transform of the sum into a product; the quadratic term survives the limit while the smaller remainder disappears.

Examples and non-examples

Let $X_j$ indicate success in independent trials with $p=0.3$. Then $S_n$ is binomial, its mean is $0.3n$, and its variance is $0.21n$. Thus

For $n=2100$, the interval has half-width $0.02$. The limiting normal probability suggests about $0.9545$, but the theorem supplies no certified error at this particular sample size.

If the observations themselves are independent $N(\mu,\sigma^2)$ variables with $\sigma>0$, then $Z_n$ is exactly standard normal for every $n$. Approximation becomes equality in this special case.

For perfect dependence $X_j=Z$ with $Z$ a fair sign, $S_n/\sqrt n=\sqrt n Z$ does not approach a normal law, despite bounded individual observations. Independence cannot simply be omitted.

Iid standard Cauchy observations fail the moment hypothesis. Their sum has the law of $nX_1$, so the $\sqrt n$ normalization does not yield a standard normal limit.

Important properties

Proof sketch: set $Y=(X_1-\mu)/\sigma$, so $E[Y]=0$ and $E[Y^2]=1$. Its characteristic function satisfies

Consequently, independence gives

The characteristic-function continuity theorem identifies the limit as standard normal. Finite second moment justifies the expansion; an infinite power-series expansion of all moments is unnecessary.

The CLT and the law of large numbers answer different questions: stabilization of an average versus the limiting distribution of its magnified error. Neither theorem guarantees that relative errors in extremely small tail probabilities are small for a chosen finite $n$.

Connections and further reading

Characteristic functions provide the main proof mechanism. The binomial specialization is the de Moivre–Laplace limit. Lawler, Proposition 5.4 and Theorem 5.5 give the proof, while Li, §4.6.2 documents 中心极限定理. Durrett, Example 3.3.16 supplies the Cauchy comparison.