Gaussian Mixture Model (GMM) & Expectation Maximization (EM)
Guassian Model
Gaussian Single Model
Guassian probability model is widely used in machine learning. The density function of single Guassian is: \[ f(x) = \frac{1}{\sqrt{2\pi}\sigma}exp(-\frac{(x-\mu)^2}{2\sigma^2}) \]
$$ is the mean and \(\sigma\) is the variance.
\(X\sim N(\mu, \sigma^2)\) means X is distributed according to N.
Multivariate Guassian Model
If \(X = (x_1, x_2, x_3, \dots, x_n)\), the density function is: \[ f(X) = \frac{1}{(2\pi)^{d/2}|\sum|^{1/2}}exp[-\frac 12(X-\mu)^T(\sum )^{-1}(X-\mu)] \]
note: \(d\) is the dimension of variable (\(x_1\)); \(\mu\) is a nx1 matrix, means of each variable.
\(\sum\) is the covariance matrix of \(X\), degree to which \(x_i\) vary together.
For a 2-D Guassian model, \(\sum = \left[ \begin{matrix} \delta_{11} & \delta_{12} \\ \delta_{21} & \delta_{22} \end{matrix} \right]\). \(\sum\) determines the shape of distribution.