On Sat, Nov 29, 2014 at 11:33 AM, Aaron Staple <aaron.sta...@gmail.com>
wrote:
> Hi Mathieu,
>
> Thanks for the information you’ve provided about the ridge implementation
> and your suggestions for scoring rankings.
>
> First off, I’d like to try and contain the scope of the project I’m
> working on. Would it be reasonable for me to implement adding get_score to
> scorer, along with the oob implementation for random forests, first? This
> discussion of the ridge code seems to have raised a new set of design and
> implementation questions that might be coded separately.
>
I understand you but we must also be careful of not creating a half-cooked
API.
>
> Also, I am coming up to speed on your suggestions regarding support for
> ranked scoring of regression predictions.
>
> My impression is that in sklearn regressors typically implement predict,
> while classifiers typically implement predict as well as predict_proba
> and/or decision_function. Currently ThresholdScorer attempts to call
> decision_function() and if that fails then it calls predict_proba(). What
> if ThresholdScorer were extended to also call predict() if neither
> decision_function or predict_proba exists? That way predict() would be
> called for regressors without an interface change for estimators.
>
This is a good suggestion. To summarize here are our options:
- introduce a new method predict_score / predict_confidence (pro:
duck-typing friendly, con: one more method)
- use isinstance(estimator, RegressorMixin) to detect regressors (pro:
simple, con: assume inheritance)
- make decision_function an alias of predict in all regressors (pro:
simple, con: can no longer detect classifiers with hasattribute(estimator,
"decision_function"))
- call predict if neither predict_proba nor decision_function are available
(pro:simple, con: can't raise an exception for classifiers with predict
only)
What do people think?
Mathieu
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
