I did the same tests as before adding fit_intercept=False and:
1. I have got the same problem as before, i.e. when I execute the RFE
multiple times I don't get the same ranking each time.
2. When I change the solver to 'sag'
(classifier_RFE=LogisticRegression(C=1e9, verbose=1, max_iter=10000,
fit_intercept=False, solver='sag')), it seems that I get the same
ranking at each run. This is not the case with the 'saga' solver.
The ranking is not the same between the solvers.
3. With C=1, it seems that I have the same results at each run for all
solvers (liblinear, sag and saga), however the ranking is not the same
between the solvers.
How can I get reproducible and consistent results?
Thanks for your help,
Best regards,
Ben
Le 24/07/2018 à 18:16, Stuart Reynolds a écrit :
liblinear regularizes the intercept (which is a questionable thing to
do and a poor choice of default in sklearn).
The other solvers do not.
On Tue, Jul 24, 2018 at 4:07 AM, Benoît Presles
<benoit.pres...@u-bourgogne.fr> wrote:
Dear scikit-learn users,
I am using the recursive feature elimination (RFE) tool from sklearn to rank
my features:
from sklearn.linear_model import LogisticRegression
classifier_RFE = LogisticRegression(C=1e9, verbose=1, max_iter=10000)
from sklearn.feature_selection import RFE
rfe = RFE(estimator=classifier_RFE, n_features_to_select=1, step=1)
rfe.fit(X, y)
ranking = rfe.ranking_
print(ranking)
1. The first problem I have is when I execute the above code multiple times,
I don't get the same results.
2. When I change the solver to 'sag' or 'saga' (classifier_RFE =
LogisticRegression(C=1e9, verbose=1, max_iter=10000), solver='sag'), it
seems that I get the same results at each run but the ranking is not the
same between these two solvers.
3. With C=1, it seems I have the same results at each run for the
solver='liblinear', but not for the solvers 'sag' and 'saga'. I still don't
get the same results between the different solvers.
Thanks for your help,
Best regards,
Ben
_______________________________________________
scikit-learn mailing list
scikit-learn@python.org
https://mail.python.org/mailman/listinfo/scikit-learn
_______________________________________________
scikit-learn mailing list
scikit-learn@python.org
https://mail.python.org/mailman/listinfo/scikit-learn
_______________________________________________
scikit-learn mailing list
scikit-learn@python.org
https://mail.python.org/mailman/listinfo/scikit-learn
