Hi Mathieu,
Thank you for your reply. If it's expensive in terms of sample size, it
totally make senses for me. However, I am still confused by the statement
in scikit-learn documents:
"It loses efficiency in high dimensional spaces – namely when the number of
features exceeds a few dozens. It might indeed give poor performance and it
loses computational efficiency."
http://scikit-learn.org/stable/modules/gaussian_process.html
Even if 'the number of features' here refers to the number of sample size,
I don't think the model would be inefficient just with sample numbers over
dozens. Could you or anyone else make it clear for me please? thanks!
Cheers,
Tao
On Mon, Mar 26, 2012 at 10:24 AM, Mathieu Blondel <[email protected]>wrote:
> If I'm not mistaken, Gaussian Processes are expensive for large n_samples,
> not for large n_features. The reason is because the kernel matrix (called
> covariance matrix in the GP literature) needs to be inversed, which takes
> O(n_samples^3) complexity with a Cholesky decomposition. That said, kernels
> methods like SVMs or Gaussian Processes are usually not used much with
> high-dimensional data. Kernels are useful to implicitly project
> low-dimensional data to higher (even infinite) dimensional spaces. If your
> data is already high-dimensional, there's nothing to gain from using
> kernels. A good example is text classification, where everyone is using
> linear kernels.
>
> HTH,
> Mathieu
>
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
