+1 for using GPs for hyperparameter search algorithms and for adding gradient-based optimization.
I would like to add that it would be nice to have less redundancy between the GP's correlation_models and the kernels in pairwise. Additionally, it would be nice if kernels could be specified externally to sklearn and passed as callables to the GP in a similar way as it is implemented in the other kernel-methods in sklearn. I am also not sure about what the method reduced_likelihood_function(theta) actually computes. I tried to combine that likelihood with a prior to choose the kernel parameters via MAP-estimation and didn't get reasonable results (computing the marginal likelihood as discussed on page 113 of the Rasmussen & Williams book and combining that with a prior fixed the issue). The documentation is also quite unclear on how the reduced-likelihood is connected to the marginal likelihood (if at all) Best, Jan On 25.11.2014 18:37, Andy wrote: > There are definitely API questions that I also just discusses with Dan. > There are some thing that we could improve, but I think the solutions > depend a lot on if we want to do kernel engineering or not. > My thinking was that this part is probably the most controversial one, > so this is what I asked about. > > I am not sure how useful our current implementation is. I have not met a > person that uses it. > That might be caused by the interface and missing documentation more > than by the fact that we don't have custom kernels, though. > > I don't think we want to add a HODLR solver (hierarchical off-diagonal > low-rank solver). > It you want that, you should probably install gorge ;) > > What I think would be great to have is gradient based optimization of > the kernel parameters, and a way to do grid-search > using data likelihood (not sure if that is currently supported). > > For the API, the naming is somewhat non-standard, it is not super clear > what the parameters mean, and it is also not super clear > whether the kernel-parameters will be optimized for a given parameters > setting. > > > > > On 11/25/2014 12:28 PM, Gael Varoquaux wrote: >> On Tue, Nov 25, 2014 at 12:23:50PM -0500, Kyle Kastner wrote: >>> specifically a HODLR solver. >> What is this. Can you tell us more? >> >>> One very specific reason to focus on GP code quality would be that it >>> opens the door to use sklearn's own code to implement some very nice >>> hyperparameter search algorithms which could be useful to many users. >> Yes. I pretty badly would like this (Kyle, are you looking for an >> internship? You know, France is nice :P). >> >> Gaël >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & more >> Get technology previously reserved for billion-dollar corporations, FREE >> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk >> _______________________________________________ >> Scikit-learn-general mailing list >> Scikit-learn-general@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > > -- Jan Hendrik Metzen, Dr.rer.nat. Team Leader of Team "Sustained Learning" Universität Bremen und DFKI GmbH, Robotics Innovation Center FB 3 - Mathematik und Informatik AG Robotik Robert-Hooke-Straße 1 28359 Bremen, Germany Tel.: +49 421 178 45-4123 Zentrale: +49 421 178 45-6611 Fax: +49 421 178 45-4150 E-Mail: j...@informatik.uni-bremen.de Homepage: http://www.informatik.uni-bremen.de/~jhm/ Weitere Informationen: http://www.informatik.uni-bremen.de/robotik ------------------------------------------------------------------------------ Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server from Actuate! Instantly Supercharge Your Business Reports and Dashboards with Interactivity, Sharing, Native Excel Exports, App Integration & more Get technology previously reserved for billion-dollar corporations, FREE http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk _______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
