Agreed. But then the setting is c=1e9 in this context (where C is the inverse regularization strength), so the regularization effect should be very small.
Probably shouldn't matter much for convex optimization, but I would still try to a) set the random_state to some fixed value b) make sure that .n_iter_ < .max_iter to see if that results in more consistency. Best, Sebastian > On Jul 24, 2018, at 11:16 AM, Stuart Reynolds <stu...@stuartreynolds.net> > wrote: > > liblinear regularizes the intercept (which is a questionable thing to > do and a poor choice of default in sklearn). > The other solvers do not. > > On Tue, Jul 24, 2018 at 4:07 AM, Benoît Presles > <benoit.pres...@u-bourgogne.fr> wrote: >> Dear scikit-learn users, >> >> I am using the recursive feature elimination (RFE) tool from sklearn to rank >> my features: >> >> from sklearn.linear_model import LogisticRegression >> classifier_RFE = LogisticRegression(C=1e9, verbose=1, max_iter=10000) >> from sklearn.feature_selection import RFE >> rfe = RFE(estimator=classifier_RFE, n_features_to_select=1, step=1) >> rfe.fit(X, y) >> ranking = rfe.ranking_ >> print(ranking) >> >> 1. The first problem I have is when I execute the above code multiple times, >> I don't get the same results. >> >> 2. When I change the solver to 'sag' or 'saga' (classifier_RFE = >> LogisticRegression(C=1e9, verbose=1, max_iter=10000), solver='sag'), it >> seems that I get the same results at each run but the ranking is not the >> same between these two solvers. >> >> 3. With C=1, it seems I have the same results at each run for the >> solver='liblinear', but not for the solvers 'sag' and 'saga'. I still don't >> get the same results between the different solvers. >> >> >> Thanks for your help, >> Best regards, >> Ben >> >> _______________________________________________ >> scikit-learn mailing list >> scikit-learn@python.org >> https://mail.python.org/mailman/listinfo/scikit-learn > _______________________________________________ > scikit-learn mailing list > scikit-learn@python.org > https://mail.python.org/mailman/listinfo/scikit-learn _______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
