Olivier Grisel wrote:
> Am I right to assume that the main reference for ELM is
http://www.ntu.edu.sg/home/egbhuang/?
Absolutely.
Lars Buitinck wrote:
> I've tried ELMs once. Apart from the hyped-up name, they're neural
> nets with random input-to-hidden weights (a very old idea) and
> least-squares fitting of the hidden-to-output weights. I could get
> some interesting results on the digits dataset
> (https://gist.github.com/larsmans/2493300) and they have the benefit
> of having two hyperparameters and very fast training, but in my experience
> the performance is very sensitive to the random seed.
It was your gist that I used as the starting point, thanks :-) They are
extremely
sensitive to the random seed.
Olivier Grisel wrote:
> Sounds like a good opportunity for ensembling then :)
This is exactly why I started my implementation - I wanted them as
candidates
in an implementation of Rich Caruana's ensemble selection method
(
http://www.cs.cornell.edu/~caruana/ctp/ct.papers/caruana.icml04.icdm06long.pdf
)
that I was writing around scikit-learn.
There are a couple of reasons that I find them interesting - they're fast,
they can use
non-differentiable heterogeneous activation functions, the solution of the
hidden-output
weights need not be simple least squares fit (although something fancier
would take away
some of the speed), and the random encoding of the inputs by the hidden
layer reminds
me of the success of random patches in the Coates & Ng work on unsupervised
feature
learning in single layer networks.
Olivier Grisel wrote:
> I guess that we won't integrate any new work that is a randomized
> version of MLP before this PR is first done implementing the
> traditional MLP learning.
Fair enough, thanks.
- David
------------------------------------------------------------------------------
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
