VBGMM is a full Bayesian estimation in both 'E-step' and 'M-step'
(although there is no such concept in VB) . The parameters in VB are random
variables, and described by a posterior distribution. The posterior
distribution is the product of the likelihood and the prior distribution.
On the other hand, although MAP estimation use the posterior distribution
as well, but it is still represented by a single value like in 'M-step'
like in EM. For example, if we use inverse Wishart distribution
W^{-1}(\Sigma|\Phi,
\nu) as the prior distribution for covariance matrix and set the
parameter \Phi
to be \alpha*I. We have \tilde{\Sigma} = \frac{n}{\nu+d+1+n}(\hat{\Sigma} +
\alpha*I)， where \hat{\Sigma} is the classic estimation of covariance
matrix. As you can see, when the number of data instances increase,
the \tilde{\Sigma}
is approximated by \hat{\Sigma}. The effect \alpha is diminished. Therefore
the effect of min_covar ( \alpha ) is not prefixed, it also depends on the
number of training data we have.
Wei
On Wed, Mar 25, 2015 at 3:18 PM, Andreas Mueller <t3k...@gmail.com> wrote:
> Thanks for your feedback.
>
> On 03/25/2015 02:59 PM, Wei Xue wrote:
>
> Thanks Andreas, Kyle, Vlad and Olivier for the detailed review.
>
> 1. For the part *Implementing VBGMM, *do you mean it would be better if
> I add specific functions to be implemented? @Andreas.
>
> I just felt the paragraph was a bit unclear, and would benefit from saying
> what exactly you want to do.
>
>
>
> 6. I would like to add a variance of EM estimation to GMM module, MAP
> estimation. Currently, the m-step use maximum likelihood estimation with
> min_covariance which prevent singular covariance estimation. I think it
> would be better to add MAP estimation for m-step, because the fixed
> min_covariance in ML estimation might be too aggressive in some cases. In
> MAP, the effect of correcting covariance will be decreasing as the number
> of data instances increases.
>
> How is this different from the VBGMM?
>
>
> 7. I would also like to add some functionality to deal with missing
> values in GMM. The situation with missing value in the training data is not
> uncommon and PRML book also mentioned that.
>
> I think this is outside the scope of this project, as we generally have
> avoided dealing with missing values in sklearn estimators directly.
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
