Dear Aleks Jakulin, Thanks a lot for your invaluable discussion on the EM and Entropy. The paper by Csiszar & Tusnady 1984 is very useful and talks exactly what I am/was confused about.
In nutshell, the KL = -(differential entropy) + sum_of (marginal_entropies) Now if I assume that my data is iid then KL = 0, these definitions are basically originating from BSS and ICA school. And according to Csiszar an application of alternating minimization, the second term in KL expansion can be considered as EM. Thanks again for pointing to the Csiszar's paper :-) Cheers, Vimal . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
