On Sat, Sep 29, 2012 at 9:39 AM, Olivier Grisel <[email protected]>wrote:
> 2012/9/29 Gael Varoquaux <[email protected]>:
> > Hey Ariel,
> >
> > On Sat, Sep 29, 2012 at 08:54:46AM -0700, Ariel Rokem wrote:
> >> Sure - here's a minimal example based on what I'm trying to do with
> this (data
> >> at the top, calculations at the bottom):
> >
> >> https://gist.github.com/3804428
> >
> > I do believe that it's a convergence problem. I have updated your gist at
> > https://gist.github.com/3804487
> > to fit with more and more iterations, and when I run it I get:
> >
> > In [1]: %run sklearn_EN_example.py
> >
> /home/varoquau/dev/scikit-learn/sklearn/linear_model/coordinate_descent.py:207:
> > UserWarning: Coordinate descent with alpha=0 may lead to unexpected
> > results and is discouraged.
> > self.positive)
> >
> /home/varoquau/dev/scikit-learn/sklearn/linear_model/coordinate_descent.py:222:
> > UserWarning: Objective did not converge for target 0, you might want to
> > increase the number of iterations
> > ' to increase the number of iterations')
> > With ElasticNet: 0.9785
> > With ElasticNet (1 refit): 0.9800
> > With ElasticNet (2 refit): 0.9809
> > With ElasticNet (3 refit): 0.9815
> > With ElasticNet (4 refit): 0.9819
> > With ElasticNet (5 refit): 0.9821
> > With ElasticNet (6 refit): 0.9823
> > With ElasticNet (7 refit): 0.9824
> > With ElasticNet (8 refit): 0.9825
> > With ElasticNet (9 refit): 0.9826
> > With ElasticNet (10 refit): 0.9827
> > With LinearRegression: 1.0000
> >
> > So the conclusion are indeed that ElasticNet with alpha=0 does not
> > converge well, as we thought. Also, the code did warn you about the
> > problem.
> >
> > The coordinate descent solver does not work well on unpenalized problem.
> > You should not use it. It's a fundemental flaw of the algorithm. One
> > algorithm cannot be well-suited for every usecase. The coordinate descent
> > solver used in the ElasticNet object is good for non-smooth problems (the
> > l1 penalty) with sparse solutions. While this is the normal setting for
> > Elastic Net, you are definitely not in this situation.
>
> I think the user warning could be improved by advising the user to
> switch to sklearn.linear_model.LinearRegression instead.
A sneakier (and probably not great) approach would be to have ElasticNet
itself try calling LinearRegression when alpha is set to 0. In a way,
that's what a user is asking for by setting alpha to 0 - give me the the
OLS solution.
Ariel
> --
> Olivier
> http://twitter.com/ogrisel - http://github.com/ogrisel
>
>
> ------------------------------------------------------------------------------
> How fast is your code?
> 3 out of 4 devs don\\\'t know how their code performs in production.
> Find out how slow your code is with AppDynamics Lite.
> http://ad.doubleclick.net/clk;262219672;13503038;z?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
------------------------------------------------------------------------------
How fast is your code?
3 out of 4 devs don\\\'t know how their code performs in production.
Find out how slow your code is with AppDynamics Lite.
http://ad.doubleclick.net/clk;262219672;13503038;z?
http://info.appdynamics.com/FreeJavaPerformanceDownload.html
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
