Testing on the global optimization problems directly will actually be a
time saver,
as they can be evaluated directly, without needing to compute an
estimator on MNIST for each point.
On 03/25/2015 03:15 PM, Gael Varoquaux wrote:
I am very afraid of the time sink that this will be.
Sent from my phone. Please forgive brevity and mis spelling
On Mar 25, 2015, at 19:47, Andreas Mueller <t3k...@gmail.com
<mailto:t3k...@gmail.com>> wrote:
I think you could bench on other problems, but maybe focus on the ones
in scikit-learn.
Deep learning people might be happy with using external tools for
optimizing.
I'd also recommend benchmarking just the global optimization part on
global optimization datasets as they were used in Jasper's work.
On 03/24/2015 06:01 PM, Christof Angermueller wrote:
Don't you think that I could also benchmark models that are
not implemented in sklearn? For instance, I could write a
wrapper DeepNet(...) with fit() and predict(), and which uses
internally theano to build a ANN? In this way, I could
benchmark complex deep networks beyond what will be possible
with the new sklearn ANN module. This might be interesting for
the deep learning community. Obvious sklearn modules to
benchmark are: * RandomForestClassifier * SVC *
GaussianProcess * Perceptron As benchmark data sets, I would
use those that were used before (see Snoek at al 2012,
Bergstra et at 2011) to evaluate optimizer like spearmint. For
classification, I candidates are * MNIST * CIFAR-10 and for
regression: * Bosting housing precises @Andy, @Kyle, and
@Matthias: thanks for your references! I will have a closer
look at them tomorrow! Christof On 20150324 21:25, Andy wrote:
One thing that might also be interesting is
"Bootstrapping" (in the compiler sense, not the statistics
sense) the optimizer. The latest Jasper Snoek paper
http://arxiv.org/abs/1502.05700 they used a
hyper-parameter optimizer to optimize the parameter of a
hyper-parameter optimiz! er on a set of optimization
tasks. https://www.youtube.com/watch?v=BIizqZ0mvIo So we
could try to optimize the parameters of the GP using the
GP :)
------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go
Parallel Website, sponsored by Intel and developed in
partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials
and more. Take a look and join the conversation now.
http://goparallel.sourceforge.net/
------------------------------------------------------------------------
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------
Dive into the World of Parallel Programming T!
he Go
Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for
all
things parallel software development, from weekly thought leadership blogs
to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now.http://goparallel.sourceforge.net/
------------------------------------------------------------------------
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
