Am 23.10.2012 09:21, schrieb Mathieu Blondel:
However, option b really has the look and smell of a method to me. Combine with the fact that some score functions need estimator-specific information (i.e.: how to retrieve unthresholded decisions), it led me to think that my favorite option would be to put as much as possible in the estimator. The option that I am championing would be to add an argument to estimators to be able to switch the score function. This argument could either be a string, say 'auc', or a score_func with a given signature (estimator specific, but we would try to have as little of these as possible).I would rather avoid adding too many parameters to the constructor that don't affect `fit`. If I fit a model, I would expect to be able to evaluate it with different metrics without creating a new estimator object. Being able to just do estimator.score(X, y, "auc") would be very convenient.
This was the idea behind https://github.com/scikit-learn/scikit-learn/pull/1198 Except that I used a function instead of a keyword - though having a keyword (and maybe optionally a function?) would also be fine. Gael had two arguments against that (iirc)- stronger API requirements on the score function (i.e. that is has to exist and that it accepts a score_func keyword) - when doing a loop over grid-searches with several estimators, not all of which support the same score_func,
the loop needs to iterate over pairs of estimators and score functions.
------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct
_______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
