On 03/06/2012 08:17 PM, Adrien wrote: > Le 06/03/2012 19:19, Andreas Mueller a écrit : > >> Hi Adrien. >> Thanks for the offer and thanks for converting people from the dark side ;) >> >> I'm not sure this is the way to go, though. >> There is already quite efficient SGD code in sklearn and this should >> probably >> be extended to handle the multi-class case. >> If you include a separate implementation, there will be a lot of code >> duplication >> and it will probably be non-trivial to get to the speed of the current >> implementation. >> > I agree with you. > > What I had in mind was just to, first, provide a simple, "stand-alone", > batch implementation of MLR for reference. Note, that I didn't find any > in python... Maybe someone else has? > > Well, I have several ;) Most of them are SGD, too.
There is also Peter's bolt, which might be a good reference implementation. http://pprett.github.com/bolt/ > Like you mentioned, this batch version will not scale very well. One > reason for this is the optimization algorithm used (scipy's BFGS in my > case). > > From then on, however, it will be easy for the SGD masters to make the > stochastic version: it will just require re-using the function to > compute the negative-log-likelihood and its gradient and replace BFGS > with SGD! > > Making the SGD handle this case is more or less the only thing that requires any real work in my opinion. Integrating the different loss functions with the current, two-class loss functions and handling 2d weights is what having multinomial logistic regression is about. The rest I can write down in <10 minutes ;) ------------------------------------------------------------------------------ Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
