On 04/06/15 20:38, Sturla Molden wrote:
> Component-wise EM (aka CEM2) is a better way of avoiding the singularity
> disease, though.
The traditional EM for a GMM proceeds like this:
while True:
global_estep(clusters)
for c in clusters:
mstep(c)
This is inherently unstable. Several clusters can become
near-singular in the M-step before there is an E-step
to redistribute the weights. You can get a "cascade of
singularities" where the whole GMM basically dies. Even
if you bias the diagonal of the covariance you still
have the basic algorithmic problem.
CEM2 proceeds like this:
while True:
for c in clusters:
estep(c)
mstep(c)
This improves stability enormously. When a cluster becomes
singular, the memberships are immediately redistributed.
Therefore you will not get a "cascade of singularities"
where the whole GMM basically dies.
Sturla
------------------------------------------------------------------------------
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
