[Statistics] Gaussian Distribution

Gaussian Distribution

Univariate

PDF

\[\begin{aligned} f(x) = \frac{1}{\sqrt{2 \pi} \sigma} e^{ - \frac{1}{2} (\frac{x - \mu}{\sigma})^2} \; (0\infty < x < \infty) \end{aligned}\]

MGF

\[\begin{aligned} M(t) = \text{exp}(\mu t + \frac{1}{2} \sigma^2 t^2) \; \end{aligned}\]

$\mathbf{Proof.}$

Consider $Z \sim N(0, 1)$. Note that for $X \sim N(\mu, \sigma^2)$, $Z = \frac{X-\mu}{\sigma}$. Then

\[\begin{aligned} \mathbb{E}[e^{tZ}] & = \int_{-\infty}^{\infty} \frac{1}{\sqrt{2 \pi}} e^{-\frac{1}{2} (z - t)^2} \cdot e^{\frac{1}{2} t^2} \cdot dz = e^{\frac{1}{2} t^2}. \end{aligned}\]

Then, for $X$,

\[\begin{aligned} \mathbb{E}[e^{tX}] & = \mathbb{E}[e^{\sigma t Z}] \cdot e^{t \mu} = e^{\mu t + \frac{1}{2} \sigma^2 t^2}. \end{aligned}\]

Mean, Variance

\[\begin{aligned} \mathbb{E}[X] &= \mu \\ \text{Var}[X] &= \sigma^2 \end{aligned}\]

Important properties

$\mathbf{Thm\ 1.1.}$ Let $X \sim N(\mu, \sigma^2)$. Then,

\[\begin{aligned} W = Z^2 = \frac{(X-\mu)^2}{\sigma^2} \sim \chi^2 (1). \end{aligned}\]

$\mathbf{Proof.}$

The CDF of $W$, $F(w)$ is

\[\begin{aligned} P(W = Z^2 \leq w) & = P(- \sqrt{w} \leq Z \leq \sqrt{w}) \\ & = \Phi (\sqrt{w}) - \Phi (- \sqrt{w}) \end{aligned}\]

As $\frac{d \Phi}{dz} = \frac{1}{\sqrt{2\pi}} e^{-\frac{1}{2} z^2}$, i.e., the PDF of standard normal, the PDF of $W$ is

\[\begin{aligned} f_W (w) & = \frac{1}{2 \sqrt{w}} \cdot \frac{1}{\sqrt{2\pi}} 2e^{-\frac{1}{2} w} \\ & = \frac{1}{\sqrt{2\pi}} w^{-\frac{1}{2}} e^{-\frac{w}{2}} \end{aligned}\]

By normalization condition, note that $\Gamma( \frac{1}{2} ) = \sqrt{\pi}$. Thus, $W \sim \chi^2(1)._\blacksquare$

$\mathbf{Thm\ 1.2.}$ Let $X_1, \cdots, X_n$ be independent random samples with $X_i \sim N(\mu_i, \sigma_i^2)$. Let $Y = \sum_{i=1}^n a_i X_i$. Then

\[\begin{aligned} Y \sim N(\sum_{i=1}^n a_i \mu_i, \sum_{i=1}^n a_i^2 \sigma_i^2). \end{aligned}\]

$\mathbf{Proof.}$

\[\begin{aligned} \mathbb{E}(e^{tY}) = \mathbb{E}(e^{t \sum_{i=1}^n a_i X_i}) = \prod_{i=1}^n e^{a_i \mu_i t + \frac{1}{2} a_i^2 \sigma_i^2 t^2}. \end{aligned}\]

Multivariate

Reference

[1] Hogg, R., McKean, J. & Craig, A., Introduction to Mathematical Statistics, Pearson 2019
[2] Wikipedia, Normal distribution