Law of Large Numbers and Central Limit Theorem

This chapter systematically studies Chebyshev’s inequality, the law of large numbers, and the central limit theorem.

Chebyshev’s Inequality

• $P(|X-E(X)|\geq \varepsilon) \leq \frac{D(X)}{\varepsilon^2}$
• Applies to random variables with any distribution

Law of Large Numbers

• Chebyshev’s law of large numbers: sample mean converges in probability to the mathematical expectation
• Bernoulli’s law of large numbers: in independent repeated trials, the frequency of an event approaches its probability
• Khinchin’s law of large numbers: sample mean of i.i.d. random variables converges in probability to the expectation

Central Limit Theorem

• De Moivre-Laplace theorem: binomial distribution converges to normal distribution
• Lindeberg-Levy theorem: the standardized sum of i.i.d. random variables converges in distribution to the normal distribution

---

Exercises

1. Use Chebyshev’s inequality to estimate the upper bound of $P(|X|\geq3)$ when $E(X)=0, D(X)=4$.
2. Explain the practical meaning of Bernoulli’s law of large numbers.
3. Let $X_i$ be i.i.d., $E(X_i)=\mu, D(X_i)=\sigma^2$, write the central limit theorem for the sample mean.
4. For binomial distribution $B(n,p)$, with large $n$ and $p$ not close to 0 or 1, write its approximate normal distribution.
5. Explain the difference between the law of large numbers and the central limit theorem.

Reference Answers

1. Chebyshev’s inequality

$P(|X|\geq3)\leq\frac{4}{9}$

---

2. Meaning of Bernoulli’s law of large numbers

In a large number of independent repeated trials, the frequency of an event approaches its probability.

---

3. Central limit theorem for sample mean

$\frac{\overline{X}_n-\mu}{\sigma/\sqrt{n}} \to N(0,1)$

---

4. Binomial distribution approximates normal distribution

$B(n,p)\approx N(np, np(1-p))$

---

5. Difference

The law of large numbers concerns the convergence of the sample mean to the expectation, the central limit theorem concerns the distribution of the sum approaching normality.