Ups I did not see the answer of Roman. Sorry about that. It is coming back to the same conclusion :)
On Wed, 9 Oct 2019 at 23:37, Guillaume Lemaître <g.lemaitr...@gmail.com> wrote: > Uhm actually increasing to 10000 samples solve the convergence issue. > SAGA is not designed to work with a so small sample size most probably. > > On Wed, 9 Oct 2019 at 23:36, Guillaume Lemaître <g.lemaitr...@gmail.com> > wrote: > >> I slightly change the bench such that it uses pipeline and plotted the >> coefficient: >> >> https://gist.github.com/glemaitre/8fcc24bdfc7dc38ca0c09c56e26b9386 >> >> I only see one of the 10 splits where SAGA is not converging, otherwise >> the coefficients >> look very close (I don't attach the figure here but they can be plotted >> using the snippet). >> So apart from this second split, the other differences seems to be >> numerical instability. >> >> Where I have some concern is regarding the convergence rate of SAGA but I >> have no >> intuition to know if this is normal or not. >> >> On Wed, 9 Oct 2019 at 23:22, Roman Yurchak <rth.yurc...@gmail.com> wrote: >> >>> Ben, >>> >>> I can confirm your results with penalty='none' and C=1e9. In both cases, >>> you are running a mostly unpenalized logisitic regression. Usually >>> that's less numerically stable than with a small regularization, >>> depending on the data collinearity. >>> >>> Running that same code with >>> - larger penalty ( smaller C values) >>> - or larger number of samples >>> yields for me the same coefficients (up to some tolerance). >>> >>> You can also see that SAGA convergence is not good by the fact that it >>> needs 196000 epochs/iterations to converge. >>> >>> Actually, I have often seen convergence issues with SAG on small >>> datasets (in unit tests), not fully sure why. >>> >>> -- >>> Roman >>> >>> On 09/10/2019 22:10, serafim loukas wrote: >>> > The predictions across solver are exactly the same when I run the code. >>> > I am using 0.21.3 version. What is yours? >>> > >>> > >>> > In [13]: import sklearn >>> > >>> > In [14]: sklearn.__version__ >>> > Out[14]: '0.21.3' >>> > >>> > >>> > Serafeim >>> > >>> > >>> > >>> >> On 9 Oct 2019, at 21:44, Benoît Presles < >>> benoit.pres...@u-bourgogne.fr >>> >> <mailto:benoit.pres...@u-bourgogne.fr>> wrote: >>> >> >>> >> (y_pred_lbfgs==y_pred_saga).all() == False >>> > >>> > >>> > _______________________________________________ >>> > scikit-learn mailing list >>> > scikit-learn@python.org >>> > https://mail.python.org/mailman/listinfo/scikit-learn >>> > >>> >>> _______________________________________________ >>> scikit-learn mailing list >>> scikit-learn@python.org >>> https://mail.python.org/mailman/listinfo/scikit-learn >>> >> >> >> -- >> Guillaume Lemaitre >> Scikit-learn @ Inria Foundation >> https://glemaitre.github.io/ >> > > > -- > Guillaume Lemaitre > Scikit-learn @ Inria Foundation > https://glemaitre.github.io/ > -- Guillaume Lemaitre Scikit-learn @ Inria Foundation https://glemaitre.github.io/
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
