On 03/07/2012 09:16 AM, Alexandre Gramfort wrote: >> For n_features> n_samples, I believe that coordinate descent is >> faster in the dual. A primal coordinate descent needs to optimize one >> w_i at a time. Therefore, if your data is high dimensional it can take >> time. Liblinear implements shrinking to avoid revisiting some >> coordinates. Maybe the greedy selection of coordinates you mention can >> also help. But then, can it be called cyclic? >> > with the sparse penalty it's faster in the primal as most w_i are zero > and you end up working on a limited set of features. > > >>> I would indeed like to see a fast coordinate descent solver for logistic >>> regression. I am more interested in the l1 penalty, but the l2 penalty is >>> also useful. Multinomial loss could fall in such work. >>> >>> For such contribution to be actually useful, I'd like the code to be >>> really fast with large n_features: we don't need a solver that doesn't >>> scale to real problem. I am not an expert, but I think that a reference >>> that I recently mentionned could be useful: >>> >>> http://www.jmlr.org/papers/volume11/yuan10c/yuan10c.pdf >>> >> Note that the coordinate decent newton (CDN) algorithms for Logistic >> Regression and L2-SVM mentioned in that paper are already in liblinear >> and hence in scikit-learn :) >> > yes but not for the multinomial. The good way is probably to use the same > strategy but that's a lot of work to patch or rewrite part of liblinear. > > Regarding adding a BFGS implementation, we've always being reluctant > to commit too naive implementation but I don't want to be dogmatic on this > if it can be useful to somebody. > > Maybe we should list on the wiki all the gist that follow the scikit API > and that might be useful. > > I love that :) Then I can finally put my MLP code somewhere ;)
------------------------------------------------------------------------------ Virtualization & Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
