Re: [Scikit-learn-general] Fast estimator for use with Recursive Feature Elimination

2012-04-13 Thread Mathieu Blondel
On Fri, Apr 13, 2012 at 11:16 PM, Mathieu Blondel wrote: > > Ridge would be a candidate but there is a bit of work to be done to > support warm start. Currently, Ridge supports two solvers: one based on > scipy.linalg.solve in the dense case and another based on > scipy.sparse.linalg.cg in the spa

Re: [Scikit-learn-general] Fast estimator for use with Recursive Feature Elimination

2012-04-13 Thread Mathieu Blondel
On Fri, Apr 13, 2012 at 10:55 PM, Conrad Lee wrote: > Thanks for the suggestion of how to change RFE.py to exploit warm_start. > Should I add this feature to rfe.py and make a pull request? Or is this > functionality too specialized? > +1 for a PR Do you know of other fast estimators I can use

Re: [Scikit-learn-general] Fast estimator for use with Recursive Feature Elimination

2012-04-13 Thread Olivier Grisel
Le 13 avril 2012 15:55, Conrad Lee a écrit : > Thanks for the suggestion of how to change RFE.py to exploit warm_start. > Should I add this feature to rfe.py and make a pull request?  Or is this > functionality too specialized? > >> As we said in the issue you opened recently, SGDRegressor doesn't

Re: [Scikit-learn-general] Fast estimator for use with Recursive Feature Elimination

2012-04-13 Thread Peter Prettenhofer
2012/4/13 Conrad Lee : > Thanks for the suggestion of how to change RFE.py to exploit warm_start. > Should I add this feature to rfe.py and make a pull request?  Or is this > functionality too specialized? > >> As we said in the issue you opened recently, SGDRegressor doesn't monitor >> convergence

Re: [Scikit-learn-general] Fast estimator for use with Recursive Feature Elimination

2012-04-13 Thread Conrad Lee
Thanks for the suggestion of how to change RFE.py to exploit warm_start. Should I add this feature to rfe.py and make a pull request? Or is this functionality too specialized? As we said in the issue you opened recently, SGDRegressor doesn't monitor > convergence therefore warm_start won't make t

Re: [Scikit-learn-general] Fast estimator for use with Recursive Feature Elimination

2012-04-13 Thread Mathieu Blondel
> Also, is the warm_start option relevant here? In theory, the coefficients > from an estimator fit with N features could be used for fitting an > estimator with N - 1 features. In practice though, RFE and RFECV might not > copy the coef_ array between runs. > > warm_start can be used but you hav

[Scikit-learn-general] Fast estimator for use with Recursive Feature Elimination

2012-04-13 Thread Conrad Lee
Hey, I've found that RFECV does a good job of selecting features, but that it's quite slow. It is quicker if it's used with a fast estimator. For example, I've found that SGDRegressor is much faster than SVR. I was wondering any regressor is faster than SGDRegressor (for dense features). Remembe