(Sorry about the comment about a revived thread, I was thinking of another
one!)
On Sun, Feb 2, 2014 at 10:43 AM, James Bergstra <james.bergs...@gmail.com>wrote:
> Glad to see this thread revived!
>
> Sklearn-users who are interested in this stuff should check out Hyperopt's
> sklearn interface:
>
> https://github.com/hyperopt/hyperopt-sklearn
>
> It's very much a work-in-progress. We're in the process of putting
> together some examples / tutorial, and a tech report that describes how
> well it works, how long it takes, etc. The results we have so far are
> encouraging...
>
> And speaking of results: we want to make the case that hyperopt-on-sklearn
> is awesome, which requires showing that it works for lots of data sets. We
> can only do so much on our own. Real use cases are a lot more interesting
> than old standard benchmarks. If someone has a dataset and they'd like to
> try hyper-optimizing their sklearn estimators & pre-processing stages, get
> in touch! Send me a private message and we can work together to make sure
> hyperopt-sklearn has what it takes for your application.
>
> Also, hyperopt's got some new algorithms on the way too... but that'll be
> the subject for another writeup.
>
> - James
>
>
> On Fri, Jan 31, 2014 at 2:22 PM, Frédéric Bastien <no...@nouiz.org> wrote:
>
>> thanks.
>>
>> Fred
>>
>> On Thu, Jan 30, 2014 at 8:28 PM, Patrick Mineault
>> <patrick.minea...@gmail.com> wrote:
>> > Sure you can:
>> >
>> > http://www.cs.toronto.edu/~jasper/bayesopt.pdf
>> >
>> > And some python code:
>> >
>> > https://github.com/JasperSnoek/spearmint
>> >
>> >
>> > On Thu, Jan 30, 2014 at 7:53 PM, Frédéric Bastien <no...@nouiz.org>
>> wrote:
>> >>
>> >> I have a question on those type of algo for hyper parameter
>> >> optimization. With a grid search, we can run all jobs in parallel. But
>> >> I have the impression that those algo remove that possibility. Is
>> >> there there way to sample many starting configuration with those algo?
>> >> But the most interresting question, if we start many jobs in parallel,
>> >> if the jobs don't finish at the same time as this happen frequently,
>> >> can we sample new test point while maximizing the "coverage" with the
>> >> currently running jobs that don't have results?
>> >>
>> >> Fred
>> >>
>> >> On Thu, Jan 30, 2014 at 5:28 PM, Gael Varoquaux
>> >> <gael.varoqu...@normalesup.org> wrote:
>> >> > On Thu, Jan 30, 2014 at 11:23:28AM -0800, James Jensen wrote:
>> >> >> Bayesian optimization is an efficient method used especially for
>> >> >> functions that are expensive to evaluate. The basic idea is to fit
>> the
>> >> >> function using Gaussian processes, using a surrogate function that
>> >> >> determines where to evaluate next in each iteration. The surrogate
>> >> >> strikes a balance between exploration (sampling intervals you
>> haven't
>> >> >> tried before) and exploitation (if previous samples in a vicinity
>> >> >> scored
>> >> >> well, then the likelihood of getting a high score in that area is
>> >> >> high).
>> >> >> Some of the math behind it is beyond me, but the general idea is
>> very
>> >> >> intuitive. Brochu, Cora, and de Freitas (2010) "A Tutorial on
>> Bayesian
>> >> >> Optimization of Expensive Cost Functions," is a good introduction.
>> >> >
>> >> >> One useful application of Bayesian optimization is hyperparameter
>> >> >> tuning.
>> >> >
>> >> > Thanks a lot for your enthousiasme and suggestion.
>> >> >
>> >> > Indeed, many of the core developpers would love to see simple
>> Bayesian
>> >> > optimization used for hyperparameter optimization, for instance
>> taking
>> >> > the gist of hyperopt https://github.com/hyperopt/hyperopt and
>> making an
>> >> > extended version of the RandomSearchCV.
>> >> >
>> >> > However there are a number of technical roadblocks to get there. In
>> >> > particular the Gaussian process could be improved (to implement
>> >> > partial_fit for online learning), and the parallel computing engine
>> >> > (joblib) does not support well as producer/consumer pattern. None of
>> >> > these problems are showstoppers, but they reduce the usefulness of a
>> >> > hyper-parameter selection object using Bayesian optimization.
>> >> >
>> >> > I would hope that we find time to implement these difficult core
>> aspects
>> >> > and eventually get to implementing a more advanced hyper-parameter
>> >> > optimizer. But all the core developers are very busy and spending a
>> lot
>> >> > of time simply maintaining the library (have a look at the number of
>> >> > issues open or pull requests that are waiting to be reviewed to have
>> an
>> >> > idea).
>> >> >
>> >> > If you want to help -beyond helping with reviewing/finishing pull
>> >> > requests and closing issues, I suggest that first, to prototype code,
>> >> > you
>> >> > could first submit an example using the Gaussian processes to do
>> >> > optimization of a noisy function. In a second step, after having that
>> >> > example merged, we could think about how to build a BayesianSearchCV
>> >> > object.
>> >> >
>> >> > Cheers,
>> >> >
>> >> > Gaël
>> >> >
>> >> >
>> >> >
>> ------------------------------------------------------------------------------
>> >> > WatchGuard Dimension instantly turns raw network data into actionable
>> >> > security intelligence. It gives you real-time visual feedback on key
>> >> > security issues and trends. Skip the complicated setup - simply
>> import
>> >> > a virtual appliance and go from zero to informed in seconds.
>> >> >
>> >> >
>> http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk
>> >> > _______________________________________________
>> >> > Scikit-learn-general mailing list
>> >> > Scikit-learn-general@lists.sourceforge.net
>> >> > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>> >>
>> >>
>> >>
>> ------------------------------------------------------------------------------
>> >> WatchGuard Dimension instantly turns raw network data into actionable
>> >> security intelligence. It gives you real-time visual feedback on key
>> >> security issues and trends. Skip the complicated setup - simply import
>> >> a virtual appliance and go from zero to informed in seconds.
>> >>
>> >>
>> http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk
>> >> _______________________________________________
>> >> Scikit-learn-general mailing list
>> >> Scikit-learn-general@lists.sourceforge.net
>> >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>> >
>> >
>> >
>> >
>> ------------------------------------------------------------------------------
>> > WatchGuard Dimension instantly turns raw network data into actionable
>> > security intelligence. It gives you real-time visual feedback on key
>> > security issues and trends. Skip the complicated setup - simply import
>> > a virtual appliance and go from zero to informed in seconds.
>> >
>> http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk
>> > _______________________________________________
>> > Scikit-learn-general mailing list
>> > Scikit-learn-general@lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> WatchGuard Dimension instantly turns raw network data into actionable
>> security intelligence. It gives you real-time visual feedback on key
>> security issues and trends. Skip the complicated setup - simply import
>> a virtual appliance and go from zero to informed in seconds.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Scikit-learn-general mailing list
>> Scikit-learn-general@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>
>
>
------------------------------------------------------------------------------
WatchGuard Dimension instantly turns raw network data into actionable
security intelligence. It gives you real-time visual feedback on key
security issues and trends. Skip the complicated setup - simply import
a virtual appliance and go from zero to informed in seconds.
http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
