Ties within a confidence interval happen in practice and it could be nice
to have grid search use a model complexity criterion to select between
insignificantly different top performers. But I think this is separate to
the notion of scorer. It relies on custom logic beyond argmax to select the
best parameters. The current design of GirdSearchCV is not particularly
suitable for extending in this way.
On 20 February 2015 at 12:41, Pagliari, Roberto <rpagli...@appcomsci.com>
wrote:
> hi Andy,
> but in practice, how do I access the estimator from the scoring function?
>
> I'm asking this because the estimator may be an SVM or random forest or
> whatever else, with different parameter names.
>
> regarding your question, it can happen with small datasets, for example,
> UCI datasets.
>
> Thank you,
>
>
> ------------------------------
> *From:* Andy [t3k...@gmail.com]
> *Sent:* Thursday, February 19, 2015 8:18 PM
>
> *To:* scikit-learn-general@lists.sourceforge.net
> *Subject:* Re: [Scikit-learn-general] custom scorer with parameters
>
> Well you could penalize by epsilon times 1/C or something, if you want a
> tradeoff of complexity and accuracy.
> You do have access to the estimator in the scoring.
>
> Or you could do it manually after doing the grid-search (with refit=False).
> Really, ties shouldn't happen in practice, though. How large is your
> dataset?
>
>
> On 02/19/2015 02:23 PM, Pagliari, Roberto wrote:
>
> With SVM, I’d really to pick the C and gamma I want when multiple pairs
> have the same best CV score..
>
>
>
> *From:* Andy [mailto:t3k...@gmail.com <t3k...@gmail.com>]
> *Sent:* Thursday, February 19, 2015 4:48 PM
> *To:* scikit-learn-general@lists.sourceforge.net
> *Subject:* Re: [Scikit-learn-general] custom scorer with parameters
>
>
>
> Not really.
> The scorer should not really know that it is inside a grid-search.
> What is it you want to do?
>
> On 02/19/2015 10:18 AM, Pagliari, Roberto wrote:
>
> I'd like to implement a custom score function for grid search CV.
>
> would it be possible to get the current best parameters from within the
> score function?
>
>
>
>
>
> ------------------------------------------------------------------------------
>
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk
>
>
>
>
> _______________________________________________
>
> Scikit-learn-general mailing list
>
> Scikit-learn-general@lists.sourceforge.net
>
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations,
> FREEhttp://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk
>
>
>
> _______________________________________________
> Scikit-learn-general mailing
> listScikit-learn-general@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
