2012/9/29 Gael Varoquaux <[email protected]>: > Hey Ariel, > > On Sat, Sep 29, 2012 at 08:54:46AM -0700, Ariel Rokem wrote: >> Sure - here's a minimal example based on what I'm trying to do with this >> (data >> at the top, calculations at the bottom): > >> https://gist.github.com/3804428 > > I do believe that it's a convergence problem. I have updated your gist at > https://gist.github.com/3804487 > to fit with more and more iterations, and when I run it I get: > > In [1]: %run sklearn_EN_example.py > /home/varoquau/dev/scikit-learn/sklearn/linear_model/coordinate_descent.py:207: > UserWarning: Coordinate descent with alpha=0 may lead to unexpected > results and is discouraged. > self.positive) > /home/varoquau/dev/scikit-learn/sklearn/linear_model/coordinate_descent.py:222: > UserWarning: Objective did not converge for target 0, you might want to > increase the number of iterations > ' to increase the number of iterations') > With ElasticNet: 0.9785 > With ElasticNet (1 refit): 0.9800 > With ElasticNet (2 refit): 0.9809 > With ElasticNet (3 refit): 0.9815 > With ElasticNet (4 refit): 0.9819 > With ElasticNet (5 refit): 0.9821 > With ElasticNet (6 refit): 0.9823 > With ElasticNet (7 refit): 0.9824 > With ElasticNet (8 refit): 0.9825 > With ElasticNet (9 refit): 0.9826 > With ElasticNet (10 refit): 0.9827 > With LinearRegression: 1.0000 > > So the conclusion are indeed that ElasticNet with alpha=0 does not > converge well, as we thought. Also, the code did warn you about the > problem. > > The coordinate descent solver does not work well on unpenalized problem. > You should not use it. It's a fundemental flaw of the algorithm. One > algorithm cannot be well-suited for every usecase. The coordinate descent > solver used in the ElasticNet object is good for non-smooth problems (the > l1 penalty) with sparse solutions. While this is the normal setting for > Elastic Net, you are definitely not in this situation.
I think the user warning could be improved by advising the user to switch to sklearn.linear_model.LinearRegression instead. -- Olivier http://twitter.com/ogrisel - http://github.com/ogrisel ------------------------------------------------------------------------------ How fast is your code? 3 out of 4 devs don\\\'t know how their code performs in production. Find out how slow your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219672;13503038;z? http://info.appdynamics.com/FreeJavaPerformanceDownload.html _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
