Wow, this is great! Thanks for all the great questions.
Sebastian Walter <sebastian.walter <at> gmail.com> writes:
> is it really the covariance matrix you want to invert? Or do you want
> to compute something like
> x^T C^{-1} x,
> where x is an array of size N and C an array of size (N,N)?
Yes, this is what I am computing. I am computing the pdf of a very high-
dimensional multivariate normal. Is there a specialized method to compute this?
> It would also be interesting to know how the covariance matrix gets computed
> and what its condition number is, at least approximately.
Chuck and Sebastian are on to me. I should look into computing this using the
small covariance matrix. I suppose I could even postpone evaluation of this
covariance matrix by passing in the data matrix instead of the covariance
matrix. What do you guys think?
As for the condition number, the matrix will, initially, be ill-conditioned
because the dimension of the data will far outnumber the number of data points.
However, I am writing this code on the way to writing code that fits a mixture
of Gaussians but can use any number of covariance regularization techniques to
"fix" the covariance matrices.
Basically, I am writing code that does this:
http://www.cs.colostate.edu/~dane/papers/ANNIE2010.pdf. I am guessing you math
and stats guys might shake your head at my clumsy, computer science ways... :)
However, in addition to the two, simple regularization techniques described in
that paper, I also want to be able to do things like mixture of factor
analyzers
and mixture of principal component analyzers and so on by simply supplying
different regularization techniques.
FYI, I am currently adding this to PyMix. Does this seem like a good package
to
extend?
> On Wed, Sep 1, 2010 at 1:58 AM, Charles R Harris
> <charlesr.harris <at> gmail.com> wrote:
> > On Tue, Aug 31, 2010 at 4:52 PM, Dan Elliott <danelliottster <at> gmail.com>
> > wrote:
> >> David Warde-Farley <dwf <at> cs.toronto.edu> writes:
> >> > On 2010-08-30, at 10:36 PM, Charles R Harris wrote:
> >> > I think he means that if he needs both the determinant and to solve the
> >> > system, it might be more efficient to do
> >> > the SVD, obtain the determinant from the diagonal values, and obtain the
> >> > solution by multiplying by U D^-1 V^T?
> >> Thank you, that is what I meant. Poorly worded on my part.
> >>
> >> In particular, I am writing code to invert a very large covariance matrix.
> >> I
> >> think David has some good information in another post in this thread.
> >>
> >
> > Where did the covariance array come from? It may be the case that you can
> > use a much smaller one, for instance in PCA of images.
> >
> > Chuck
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
