Another point of confusion:
You shouldn't be using clf.predict() to calculate roc auc, you need
clf.predict_proba(). Roc is a measure of sorting and predict only gives you
the predicted class, not the probability, so the roc "curve" can only have
points at 0 and 1 instead of any probability in between.
On Thu, May 12, 2016 at 9:15 AM, Andreas Mueller <t3k...@gmail.com> wrote:
> How did you evaluate on the development set?
> You should use "best_score_", not grid_search.score.
>
>
> On 05/12/2016 08:07 AM, A neuman wrote:
>
> thats actually what i did.
>
> and the difference is way to big.
>
> Should I do it withlout gridsearchCV? I'm just wondering why gridsearch
> giving me overfitted values. I know that these are the best params and so
> on... but i thought i can skip the manual part where i test the params on
> my own. GridsearchCV give me just one pool of params, if they are
> overfitting, i cant use gridsearchCV? Just having problems to understand
> this.
>
>
>
> On 12 May 2016 at 13:45, Olivier Grisel <olivier.gri...@ensta.org> wrote:
>
>> 2016-05-12 13:02 GMT+02:00 A neuman < <themagenta...@gmail.com>
>> themagenta...@gmail.com>:
>> > Thanks for the answer!
>> >
>> > but how should i check that its overfitted or not?
>>
>> Do a development / evaluation split of your dataset, for instance with
>> the train_test_split utility first. Then train your GridSearchCV model
>> on the development set and evaluate it both on the development set and
>> on the evaluation set. If the difference is large it means that you
>> are overfittng.
>>
>> --
>> Olivier
>>
>>
>> ------------------------------------------------------------------------------
>> Mobile security can be enabling, not merely restricting. Employees who
>> bring their own devices (BYOD) to work are irked by the imposition of MDM
>> restrictions. Mobile Device Manager Plus allows you to control only the
>> apps on BYO-devices by containerizing them, leaving personal data
>> untouched!
>> https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
>> _______________________________________________
>> Scikit-learn-general mailing list
>> Scikit-learn-general@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>
>
>
>
> ------------------------------------------------------------------------------
> Mobile security can be enabling, not merely restricting. Employees who
> bring their own devices (BYOD) to work are irked by the imposition of MDM
> restrictions. Mobile Device Manager Plus allows you to control only the
> apps on BYO-devices by containerizing them, leaving personal data
> untouched!https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
>
>
>
> _______________________________________________
> Scikit-learn-general mailing
> listScikit-learn-general@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
>
>
> ------------------------------------------------------------------------------
> Mobile security can be enabling, not merely restricting. Employees who
> bring their own devices (BYOD) to work are irked by the imposition of MDM
> restrictions. Mobile Device Manager Plus allows you to control only the
> apps on BYO-devices by containerizing them, leaving personal data
> untouched!
> https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
------------------------------------------------------------------------------
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
