Hi David. Thanks for your email. I would appreciate it if we could keep the discussion on the list for as long as possible, as other people might have something to say or are interested.
I recently posted a gist: https://gist.github.com/2061456 And there is also a branch by me: https://github.com/amueller/scikit-learn/tree/multilayer_perceptron These would be good places to start. This implements SGD. My biggest wish would be to do the SGD in Cython, which should result in a major speedup. This can probably be done using the recently used helper classes from the linear SGD, which might need to be extended. As far as deep learning goes: I agree, this is to much and I think it is at the moment not in the scope of the scikit. Apart from writing the SGD, there should be support for differnt loss-functions. In my current implementation, the hinge-loss support is not working yet. I feel it hard to judge the amount of work necessary since I am not so experienced in Cython myself, but I think it should be possible to extent SGD to other optimization methods. Levenberg-Marquardt is definitely interesting and would be good to have. Maybe you can compare it with resilient backpropagation and also include that if it proves helpful. I would like to hear from the "SGD people" what they think about this project and what they think is doable in the time. Cheers, Andy ps: I would be interested in hearing what parts of sklearn your advisor uses :) On 03/19/2012 01:57 PM, David Marek wrote: > Hi Andreas, > > My name is David Marek. I am studying for Master's Degree in > Theoretical Computer Science at Faculty of Mathematics and Physics, > Charles University in Prague. I would like to apply for gsoc and work > on scikit-learn. I like the Neural Networks idea. I have experience in > using different learning algorithms in Matlab and I have written > simple backpropagation algorithm. > > What do you think the soc project should contain? My idea is to create > one estimator for feed forward neural networks classification and one > for regression with different methods available for learning. One is > the stochastic gradient descent algorithm. However, I am not sure that > project would be big enough. Maybe add another learning algorithms, > e.g. some second order learning algorithm (Levenberg-Marquardt is > quite popular in Matlab although it isn't very usable on large > dataset). What do you think? On the other hand there would be a lot of > work on documentation and creating new examples and tests. I am not > sure how much time would this project take. > > If I should specify what my idea doesn't contain, it's deep learning. > I think that's beyond the scope of this idea. > > I would really like to work on scikit-learn because I am probably > going to use it in my Master's thesis which will be about statistical > methods for spoken dialogue management. I found scikit-learn when my > advisor told me he uses it for his research in machine learning. > However, I am not sure I am going to be able to participate in gsoc > right now, because I have also applied for google internship and > passed all interviews but haven't received any information about > available positions yet. I sent my application too late so I am not > sure there will be any place available for me and in that case I would > like to do gsoc. I have been told I should get the final verdict > before 26th March which means I should know if I'll be able to > participate before the student application opens. So that's a little > warning. > > Thanks > > David Marek > ------------------------------------------------------------------------------ This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
