I'd be happy to help define and mentor this PR, if a mentor is needed. I'd
really like to see this nnet work merged into sklearn, and some of the
other ideas that have been mentioned here too (e.g. docs & code for ELM).
On Wed, Feb 26, 2014 at 10:56 AM, federico vaggi
<vaggi.feder...@gmail.com>wrote:
> As an aside Lars - I'd actually love to see the recepy, if you don't mind
> putting up a gist or notebook.
>
>
> On Wed, Feb 26, 2014 at 1:29 PM, Lars Buitinck <larsm...@gmail.com> wrote:
>
>> 2014-02-25 7:52 GMT+01:00 Gael Varoquaux <gael.varoqu...@normalesup.org>:
>> >> Extreme learning machine: theory and applications has 1285 citations
>> >> and it got published in 2006; a large number of citations for a fairly
>> >> recent article. I believe scikit-learn could add such an interesting
>> >> learning algorithm along with its variations (weighted ELMs, sequential
>> >> ELMS, etc.)
>> >
>> > It does sound like a possible candidate for inclusion.
>>
>> We have a PR that implements them, but in too convoluted a way. My
>> personal choice for implementing these would be a transformer doing a
>> random projection + nonlinear activation. That way, you can stack any
>> linear model on top (think SGDClassifier for large-scale work) and get
>> a basic ELM. I've toyed with this variant before (typing this from
>> memory):
>>
>> class RandomHiddenLayer(BaseEstimator, TransformerMixin):
>> def __init__(self, n_components=100, random_state=None):
>> self.n_components = n_components
>> self.random_state = random_state
>> def fit(self, X, y=None):
>> random_state = check_random_state(self.random_state)
>> self.components_ = random_state.randn(n_components, X.shape[1])
>> return self
>> def transform(self, X):
>> return np.tanh(safe_sparse_dot(X, self.components_.T))
>>
>> Now, make_pipeline(RandomHiddenLayer(), SGDClassifier()) is an ELM
>> except with regularized hinge loss instead of least squares. I guess
>> LDA can be used to get the "real" ELM.
>>
>> I recently implemented baseline RBF networks in pretty much the same
>> way: k-means + RBF kernel + linear classifier. I didn't submit a PR
>> because it's just a pipeline of existing components.
>>
>> >> Chances are the Multi-layer perceptron PR would be completed before the
>> >> summer, so it won't be included in the GSoC proposal.
>> >
>> >> In order not to get into a scope creep, I compiled the following list
>> of
>> >> algorithms to be proposed for the GSoC 2014,
>> >
>> >> 1) Extreme Learning Machines (
>> http://sentic.net/extreme-learning-machines.pdf)
>> >> 1a) Weighted Extreme Learning Machines
>> >> 1b) Sequential Extreme Learning machines
>>
>> Does sequential mean for sequence data?
>>
>>
>> ------------------------------------------------------------------------------
>> Flow-based real-time traffic analytics software. Cisco certified tool.
>> Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
>> Customize your own dashboards, set traffic alerts and generate reports.
>> Network behavioral analysis & security monitoring. All-in-one tool.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Scikit-learn-general mailing list
>> Scikit-learn-general@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>
>
>
>
> ------------------------------------------------------------------------------
> Flow-based real-time traffic analytics software. Cisco certified tool.
> Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
> Customize your own dashboards, set traffic alerts and generate reports.
> Network behavioral analysis & security monitoring. All-in-one tool.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
------------------------------------------------------------------------------
Flow-based real-time traffic analytics software. Cisco certified tool.
Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
Customize your own dashboards, set traffic alerts and generate reports.
Network behavioral analysis & security monitoring. All-in-one tool.
http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
