Central Limit Theorem

A Most Magical Theorem ✨. It says two things:

1. The mean of IID Random Variables $X_i$ sampled from any distribution is Normally Distributed.

$$\bar{X} = \frac{1}{n}\sum_{i=1}^{n}{X_i} \sim N(\mu, \frac{\sigma^2}{n}) \space \text{as} \space n \rarr \infty$$

2. The sum of equally weighted IID Random Variables $X_i$ from any distribution is also Normally Distributed.

$$n\bar{X} = n\sum_{i=1}^{n}{X_i} \sim N(n\mu, n\sigma^2) \space \text{as} \space n \rarr \infty$$

How nice. The underlying distribution doesn't matter!

People like expressing #1 as the Standard Normal $Z$, a beautiful little bell-curve with its mean at $0$ and Standard Deviation $1$. Here's the general form:

$$Z = \frac{X - \mu}{\sigma}$$

and a form relevant to what we're talking about. The denominator normalizes $Z$ to $[0,1]$.

$$Z = \frac{\bar{X} - \mu}{\frac{\sigma}{\sqrt{n}}}$$

Why the Normal?

Because of the Law of Large Numbers¹ plus the fact that the normal distribution is the only stable distribution under addition (when variance is finite). When you add up a bunch of things, the heres and theres, ups and downs, lefts and rights tend to sort of cancel out. The only thing that survives... (TODO This is akin to asking "Why is TV noise Gaussian?")

Footnotes

1. When $n$ is sufficiently large and Random Variable $X$ is IID, the sample mean $\bar{X}$ approaches ("converges to") the true, population mean $\mu$. ↩