EM Algorithm

目录

• EM Algorithm
  • Jensen’s inequality
  • EM Algorithm

Jensen’s inequality

convex function: /(f”(x) /ge 0/) or /(H /ge 0/) (Hessian matrix when x is a vector)

/[E[f(x)] /ge f(EX)
/]

EM Algorithm

EM can be proved that it make the likelihood function increase monotonically.

maximize the lower-bound on the likelihood /(/ell/), /(/log/) is a concave function. the process is

/[/theta_{t-1} /rightarrow^{assign} Q_{z,t-1} /rightarrow^{/arg/max} /theta_{t} /rightarrow Q_{z,t}
/]

define:

/[J(Q,/theta)=/sum_{i} /sum_{z^{(i)}}Q_i(z^{(i)})/log /frac{p(x^{(i)},z^{(i)};/theta)}{Q_i(z^{(i)})}
/]

the EM algorithm can also be defined as coordinate ascent on /(J/).