Thanks for all the responses! Indeed, taking a precomputed kernel matrix would be extremely useful to us since some of the terms in the kernel expression take some time to generate for the whole original training set. Unfortunately I don't know if GPs could actually work like that -- do they need to measure correlations between x outside of the training set?
We've already been using Mathieu's kernel ridge branch for some time, but would like to investigate the performance of GPs for our problem since it appears to give better performance even with the default squared exponential correlation. A collaborator of ours has had good success with a physics-inspired similarity kernel (see section IV.A, pg 10 of [1]) that may or may not be stationary. Unfortunately, I've been told that due to possible licensing problems it is not advisable to look at their code for reference. (By the way, I'm working under the assumption that "similarity kernel" and "correlation kernel" are the same thing, so sorry if that's not how it goes!) Mathieu: I did not understand your comment about the gist being "essentially kernel ridge." Do you mean that KRR is a simpler special case of GP? In what way does that gist differ from the implementation in your KRR branch? Vincent: I did not mean to imply that sklearn's usage of K(theta, dx) was in any way unusual, just that the first few hits for correlation functions on google (and some in the Rasmussen & Williams book) were in the more general k(x, x') form. [1] - http://arxiv.org/abs/1209.3140 2013/12/10 Vincent Dubourg <vincent.dubo...@gmail.com>: > Hi Ralf, Peter, > > sklearn's implementation of GPML already enables custom correlation function > as far as they are callables with signature matching corr(theta, dx). > > I can't see anything unusual in the arguments of the sklearn correlation > function. GPML theory (or at least what the GaussianProcess estimator > implements) is restricted to stationary correlation functions, so there is > no need to take x and x' as arguments. The Manhattan distance d between x > and x' is enough, isn't it? > > Regarding the spherical Bessel function, did you have a look at scipy > specials? (http://docs.scipy.org/doc/scipy-0.7.x/reference/special.html) > > If you're trying to implement the Matern kernel, import the matern function > from the attached python file. > > Cheers, > Vincent > > > > 2013/12/10 Mathieu Blondel <math...@mblondel.org> >> >> Hi Ralf, >> >> You might find the following gist useful: >> https://gist.github.com/mblondel/6230778 >> >> It should support the usual kernel="precomputed". However, my >> implementation doesn't support the usual features that one would expect from >> a GP implementation such as kernel parameter tuning or ARD. In fact, this is >> essentially a kernel ridge implementation. >> >> HTH, >> Mathieu >> >> >> >> On Tue, Dec 10, 2013 at 5:53 PM, Peter Prettenhofer >> <peter.prettenho...@gmail.com> wrote: >>> >>> Hi Ralf, >>> >>> unfortunately, I cannot answer your question but it would be indeed very >>> valuabe to allow custom correlation functions. >>> >>> best, >>> Peter >>> >>> >>> 2013/12/9 Ralf Gunter <ralfgun...@gmail.com> >>>> >>>> Hi all, >>>> >>>> We're trying to use a custom correlation kernel with GP in the usual >>>> form K(x, x'). However, by looking at the built-in correlation models >>>> (and how they're used by gaussian_process.py) it seems sklearn only >>>> takes models in the form K(theta, dx). There may very well be a >>>> reformulation of our K that depends only on (x-x'), but if so it would >>>> probably be highly non-trivial as it depends on e.g. modified >>>> spherical bessel functions evaluated at a scaled product of the xs. Is >>>> there any way to have the GP module take our kernel without modifying >>>> the GP code? >>>> >>>> I apologize if this has been asked/answered before -- some searching >>>> on google only led me to models that also depend only on (x-x'). >>>> >>>> Thanks! >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Sponsored by Intel(R) XDK >>>> Develop, test and display web and hybrid apps with a single code base. >>>> Download it for free now! >>>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk >>>> _______________________________________________ >>>> Scikit-learn-general mailing list >>>> Scikit-learn-general@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>> >>> >>> >>> >>> -- >>> Peter Prettenhofer >>> >>> >>> ------------------------------------------------------------------------------ >>> Sponsored by Intel(R) XDK >>> Develop, test and display web and hybrid apps with a single code base. >>> Download it for free now! >>> >>> http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Scikit-learn-general mailing list >>> Scikit-learn-general@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>> >> >> >> >> ------------------------------------------------------------------------------ >> Sponsored by Intel(R) XDK >> Develop, test and display web and hybrid apps with a single code base. >> Download it for free now! >> >> http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk >> _______________________________________________ >> Scikit-learn-general mailing list >> Scikit-learn-general@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> > > > ------------------------------------------------------------------------------ > Rapidly troubleshoot problems before they affect your business. Most IT > organizations don't have a clear picture of how application performance > affects their revenue. With AppDynamics, you get 100% visibility into your > Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics > Pro! > http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > ------------------------------------------------------------------------------ Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk _______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
