On Fri, Oct 05, 2012 at 09:56:00AM +0200, Gael Varoquaux wrote:
> This is a fundamental problem of the lasso path, that can have an huge
> number of kinks: http://arxiv.org/abs/1205.0079 computing the full path
> can thus be very complex, and accumulate numerical errors.
By the way, out of the above paper, that I believe is the state of the
art in terms of stability of homotopy methods for Lasso, I found the
following sentence (page 3, first column):
"""When (X_J^T X^J) becomes ill-conditioned, which may typically occur for
small values of lambda, the algorithm has to stop and the path is
truncated."""
In other word: give up when the sub-problem becomes too ill-conditioned.
I guess that's a pragmatic solution. It might be what R is doing. It does
make sens: when lambda becomes too small, the problem is not regularized
enough, and it cannot be solved well. The solution is not even of a
pragmatic use, as it won't be a good estimator or a good generalization
model.
I have personally suffered from the same problem: basically Lars blowing
up either numerically, or even worst running into overflows and crashing.
I would be very happy including the above solution in scikit-learn.
What do people think?
Gaël
------------------------------------------------------------------------------
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p.sf.net/sfu/newrelic-dev2dev
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
