I went through the paper, for me the take home points are the following:
The comparison has to be taken with a grain of salt. The GLMNET
implementation used in the
comparison was v.1.5.3 while the current version (1.6) has new stopping
rules and uses strong rules to early discard features. [2]
Better stopping rules and line search are some of the improvements in
newGLMNET.
The authors of [1] acknowledge that "Preliminary results in the
supplementary
document show that training is more efficient if we can remove some zero
variables beforehand.
How to efficiently and correctly identify these variables before training
is an interesting future topic."
newGLMNET is used as the L1-regularized logistic regression solver in
LIBLINEAR [1]
What's the main point of implementing regularized log loss in scikit-learn?
How about a straight forward implementation
with strong rules that is well integrated in scikit-learn?
It appears to me that lot's of tricks are needed to stand a chance to be
competitive with
the fastest implementations (lots of Cython pointer kung fu etc.). I got
burned already by a innocent looking caching trick
in the GLMNET penalized L2 loss implementation.
I'm going to have a look a the LIBLINEAR code now.
Immanuel
[1] Yuan, G. X, C. H Ho, and C. J Lin. “An Improved GLMNET for
L1-regularized Logistic Regression and Support Vector Machines.” *National
Taiwan University, Tech. Rep* (2011).
http://w.csie.org/~cjlin/papers/l1_glmnet/long-glmnet.pdf.
[2]
http://r.789695.n4.nabble.com/R-pkgs-glmnet-1-6-uploaded-to-CRAN-td3461975.html
2012/7/16 Alexandre Gramfort <[email protected]>
> >> if it is I would seriously consider patching Liblinear to allow warm
> >> restart than starting a complete reimplementation.
> >
> > I guess that part of your message is that backporting the latest
> > liblinear to run speed tests might be important.
>
> yes and eventually give up on a complete reimplementation.
>
> Alex
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
