I decided to only benchmark scikit-learn models. Doing this properly and
summarizing the results in a user-friendly rst document will take some
time and should be sufficient for a GSoC project. More sophistacted
benchmarks could be carried out afterwards.
I plan to benchmark the following models:
* RandomForestClassifer
* SVC
* the the MLP
Or is there another model which I should include?
I will use some of the datasets described in the spearmint publication,
including
* MNIST,
* CIFAR-10, and
* Bosting housing prices.
Christof
I decided to only benchmark scikit-learn models.
On 20150325 19:42, Andreas Mueller wrote:
Testing on the global optimization problems directly will actually be
a time saver,
as they can be evaluated directly, without needing to compute an
estimator on MNIST for each point.
On 03/25/2015 03:15 PM, Gael Varoquaux wrote:
I am very afraid of the time sink that this will be.
Sent from my phone. Please forgive brevity and mis spelling
On Mar 25, 2015, at 19:47, Andreas Mueller <t3k...@gmail.com
<mailto:t3k...@gmail.com>> wrote:
I think you could bench on other problems, but maybe focus on the ones
in scikit-learn.
Deep learning people might be happy with using external tools for
optimizing.
I'd also recommend benchmarking just the global optimization part on
global optimization datasets as they were used in Jasper's work.
On 03/24/2015 06:01 PM, Christof Angermueller wrote:
Don't you think that I could also benchmark models that are
not implemented in sklearn? For instance, I could write a
wrapper DeepNet(...) with fit() and predict(), and which uses
internally theano to build a ANN? In this way, I could
benchmark complex deep networks beyond what will be possible
with the new sklearn ANN module. This might be interesting
for the deep learning community. Obvious sklearn modules to
benchmark are: * RandomForestClassifier * SVC *
GaussianProcess * Perceptron As benchmark data sets, I would
use those that were used before (see Snoek at al 2012,
Bergstra et at 2011) to evaluate optimizer like spearmint.
For classification, I candidates are * MNIST * CIFAR-10 and
for regression: * Bosting housing precises @Andy, @Kyle, and
@Matthias: thanks for your references! I will have a closer
look at them tomorrow! Christof On 20150324 21:25, Andy wrote:
One thing that might also be interesting is
"Bootstrapping" (in the compiler sense, not the
statistics sense) the optimizer. The latest Jasper Snoek
paper http://arxiv.org/abs/1502.05700 they used a
hyper-parameter optimizer to optimize the parameter of a
hyper-parameter optimiz! er on a set of optimization
tasks. https://www.youtube.com/watch?v=BIizqZ0mvIo So we
could try to optimize the parameters of the GP using the
GP :)
------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go
Parallel Website, sponsored by Intel and developed in
partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials
and more. Take a look and join the conversation now.
http://goparallel.sourceforge.net/
------------------------------------------------------------------------
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------
Dive into the World of Parallel Programming T!
he Go
Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for
all
things parallel software development, from weekly thought leadership blogs
to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now.http://goparallel.sourceforge.net/
------------------------------------------------------------------------
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now.http://goparallel.sourceforge.net/
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
--
Christof Angermueller
cangermuel...@gmail.com
http://cangermueller.com
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
