Not to bandwagon extra things on this particular effort, but one future
consideration is that if scikit-learn supported multilayer neural networks,
and eventually multilayer convolutional neural networks, it would become
feasible to load pretrained nets ALA OverFeat, DeCAF (recent papers with
sweet results) and use them as transforms.
I am doubtful about the ability to train a reasonably deep neural network
without GPU, specialized hardware, or a server, but I think loading
pretrained coefficients and using them as a transform is very reasonable.
It may be too "messy" for adoption in scikit-learn, but an adapter layer
could be very useful - I know this is basically what I and other
competitors used for a recent kaggle competition with great success.
Kyle
On Wed, Feb 5, 2014 at 9:30 AM, Gael Varoquaux <
[email protected]> wrote:
> On Wed, Feb 05, 2014 at 03:02:24PM +0300, Issam wrote:
> > I have been working with scikit-learn for three pull requests - namely,
> > Multi-layer Perceptron (MLP), Sparse Auto-encoders, and Gaussian
> > Restricted Boltzmann Machines.
>
> Yes, you have been doing good work here!
>
> > For the upcoming GSoC, I propose to ensure completing these three pull
> > requests. I also would develop Greedy layer-wise training algorithm for
> > deep learning, extending MLP to allow for more than one hidden layer,
> > where weights are initialized using Sparse Auto-encoders or RBM.
>
> > How will this suit for GSoC?
>
> The MLP is almost finished. I would hope that it would be finished before
> the GSoC. Actually, I was hoping that it could be finished before next
> release.
>
> For the rest, I'll let someone who knows neural nets better than me
> reply, as I don't know the state of the art, and I don't know what is
> feasible in deep learning without GPUs.
>
>
> Cheers,
>
> Gaël
>
>
> ------------------------------------------------------------------------------
> Managing the Performance of Cloud-Based Applications
> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> Read the Whitepaper.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
------------------------------------------------------------------------------
Managing the Performance of Cloud-Based Applications
Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
Read the Whitepaper.
http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
