On 03/10/2013 02:04 PM, Joel Nothman wrote: > > > The only output of the ``scoring`` parameter that is currently stored > in GridSearchCV.cv_scores_ is the score. But the score is also > aggregated (by BaseSearchCV to produce mean_validation_score), which > requires that it can be added to 0, accumulated with += and, if > BaseSearchCV.iid=True, divided with /=. This means that the Scorer > cannot currently return an arbitrary score object: it needs to be a > numeric type. > > Alternatively, a custom Scorer could indeed store data every time it > is called, but it would not be stored in or in correspondence with > GridSearchCV.cv_scores_. > I completely agree, this second method was what I meant. > I hope my issue is a bit clearer, but perhaps I need to implement a > possible solution to make it clearer. > Yes, your issue is clear. I am not sure what the best solution would be, though, and I am not sure we should implement one. This seems somewhat of a special case and not really the use case of GridSearchCV.
What you can always do is use IterGrid and iterate over the parameters yourself and store whatever information you'd like. We are currently in the process of making cv_scores_ more powerful, by using named tuples and storing training scores and training times. Being able to store arbitrary additional information sounds like it would complicate the interface of the scoring and the GridSearchCV quite a bit, and I'm not sure it is worth it. If you have an elegant solution, I'm all ears, though ;) Cheers, Andy ------------------------------------------------------------------------------ Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
