I read with puzzlement this discussion, because EM theoretically will never lead to increases and decreases because it can never decrease the likelihood. The only practical case where it might appear to do so is with round-off error due to finite accuracy of computers, but in over a quarter century of experience, I've never seen a real example of this. When it was claimed, it always turned out to be a programming bug of some kind.
Re 1. SAS now says that its EM is not EM but is some other algorithm, which they have called EM? I didn't know there was another algorithm for maximizing likelihoods called EM -- do they have a reference? Re 3. I do not follow what "overly strict convergence criterea" could mean, except perhaps being more strict than the machine could tolerate, and then the algorithm would "randomly" converge someplace within this tolerance, I think. By memory, your examples do not appear challenging enough to expose such possible problems. DBR On Wed, 23 Jul 2003, Fay Hughes wrote: > Many thanks for the advice and suggestions, after 2 weeks of email > conversations with SAS, here are a few points you may find interesting: > > 1. It is possible for SAS -2logL to increase and decrease, in an attempt > to find the minimum (unlike what I expected) - I suspect that for some > computer reason (storage space, speed etc), SAS is therefore not > implementing the EM algorithm as originally formulated - although I have > been unable to get clarity on this point. > > 3. In my particular case the EM algorithm converges to the global > minimum and then, apparently due to over strict convergence criteria, > proceeds to move away from that point to another - in fact a local > minimum, this convergence is also probably not helped by, most likely, > an oddly shaped loglikelihood. > > In conclusion, I suspect that careful inspection of the iteration > history would be a worthwhile undertaking, as in my case the result SAS > yields is not the global minimum, but rather a local one. Another way of > checking convergence, through different starting values for the EM is > not implemented in Version 8.2, but SAS informs me that it is an option > in version 9. > > Many thanks for any help, > Fay Hosking (nee Hughes) > Statistics MSc Student > School of Mathematical and Statistical Sciences > University of Natal, South Africa > -- Donald B. Rubin John L. Loeb Professor of Statistics Chairman Department of Statistics Harvard University Cambridge MA 02138 Tel: 617-495-5498 Fax: 617-496-8057
