Hi Christof.
I think implementing either the GP or SMAC approach would be good.
I talked to Jasper Snoek on Friday, possiblity the trickiest part for
the GP is the optimization of the resulting function.
Spearmint also marginalizes out the hyperparameters, which our upcoming
GP implementation doesn't support afaik.
I haven't looked into SMAC too deeply yet, but the main issue there is
The idea behind this project is as Kyle says to have something that is
easily accessible and integrates with scikit-learn,
as a replacement for GridSearchCV or RandomizedSearchCV. Btw, "old"
Spearmint is GPL,
"new" spearmint is under a non-commercial license.
Best,
Andy
On 03/07/2015 07:39 AM, Christof Angermueller wrote:
Hi Andreas (and others),
I am a PhD student in Bioinformatics at the University of Cambridge,
(EBI/EMBL), supervised by Oliver Stegle and Zoubin Ghahramani. In my
PhD, I apply and develop different machine learning algorithms for
analyzing biological data.
There are different approaches for hyperparameter optimization, some
of which you mentioned on the topics page:
* Sequential Model-Based Global Optimization (SMBO) ->
http://www.cs.ubc.ca/labs/beta/Projects/SMAC/
* Gaussian Processes (GP) -> Spearmint;
https://github.com/JasperSnoek/spearmint
* Tree-structured Parzen Estimator Approach (TPE) -> Hyperopt:
http://hyperopt.github.io/hyperopt/
And more recent approaches based on neural networks:
* Scalable Bayesian Optimization Using Deep Neural Networks Deep
Networks for Global Optimization (DNGO) -> http://arxiv.org/abs/1502.05700
The idea is to implement ONE of this approaches, right?
Do you prefer a particular approach due to theoretical or practical
reasons?
Spearmint also supports distributing jobs on a cluster (SGE). I
imagine that this requires platform specific code, which could be
difficult to maintain. What do you think?
Spearmint and hyperopt are already established python packages.
Another sklearn implementation might be considered as redundant, are
hard to establish. Do you have a particular new feature in mind?
Cheers,
Christof
--
Christof Angermueller
cangermuel...@gmail.com
http://cangermueller.com
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
