On Fri, Mar 21, 2014 at 11:19 AM, Issam <issamo...@gmail.com> wrote:
> >
> > Also: new topic: Did you mention earlier in the thread that you need
> > derivatives to implement a regularized ELM? Why don't you just use
> > some of the existing linear (or even non-linear?) regression models in
> > sklearn to classify the features computed by the initial layers of the
> > ELM? This is a more detailed question that doesn't really affect your
> > proposal, but I'd like to hear your thoughts and maybe discuss it.
> >
> " regression models in sklearn to classify the features computed by the
> initial layers of the ELM" I didn't get this, do you mean we can use PCA
> or SVDs to get more meaningful hidden features?
>
> No that's not what I meant. Maybe we can chat about this off-list after I
read more about ELMs e.g. the reference you gave below. And maybe your
Masters thesis too?
> The derivative is for solving the dual optimization problem by the KKT
> theorem, allowing us to add constraints like in SVM.
> Please look at page 6 in,
> http://www.ntu.edu.sg/home/egbhuang/pdf/ELM-Unified-Learning.pdf
>
>
Thanks for the reference!
> > Sounds good, but I wouldn't be so confident they always take seconds
> > to train. I think some deep vision system models are pretty much just
> > big convolutional ELMs (e.g.
> > http://jmlr.org/proceedings/papers/v28/bergstra13.pdf) and they can
> > take up to say, an hour of GPU time to (a) compute all of the features
> > for a big data set and (b) train the linear output model. Depending on
> > your data set you might want to use more than 150 output neurons! When
> > I was doing those experiments, it seemed that models got better and
> > better the more outputs I used, they just take longer to train and
> > eventually don't fit in memory.
> True, well "seconds to train" was in its figurative sense. However, the
> time it takes is nothing like backpropagation ;). You could go as large
> as 1000 hidden neurons and time is still not an issue. It wouldn't be
> slower than SVM, for example. The real issue lies in memory :).
>
> Yep, that's right. Not necessarily trivially fast, but way faster than
backprop. A pretty normal computer should be fine for the ELM and even the
backprop stuff when you get to it.
------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
