Here's a comparison of all of them: EVALUATION: FROM PRECISION, RECALL AND
F-MEASURE TO ROC, INFORMEDNESS, MARKEDNESS & CORRELATION
<http://dspace2.flinders.edu.au/xmlui/bitstream/handle/2328/27165/Powers%20Evaluation.pdf>
I warmly recommend MCC, though lots of people still use ROC
On Wed, Jul 23, 2014 at 6:09 AM, Joel Nothman <joel.noth...@gmail.com>
wrote:
> Precision, Recall and F-measure are often contrasted with Accuracy in
> terms of their handling imbalance. I'm sure I could find a textbook
> citation, but for an online example Chris Manning thus introduces P/R/F in
> the imbalanced spam classification problem on coursera:
> https://class.coursera.org/nlp/lecture/142.
>
>
> On 23 July 2014 11:35, Mathieu Blondel <math...@mblondel.org> wrote:
>
>> AUC (area under the roc curve) is commonly used for imbalanced binary
>> classification problems.
>> The AUC is the probability that your classifier will rank a positive
>> sample higher than a negative sample (where the ranking is computed using
>> the "decision_function" scores).
>> In scikit-learn, it is implemented in roc_auc_score.
>>
>> Mathieu
>>
>>
>> On Wed, Jul 23, 2014 at 12:26 AM, Hamed Zamani <hamedzam...@acm.org>
>> wrote:
>>
>>> Hi,
>>>
>>> I am working on a binary classification problem in which both training
>>> and test data are highly imbalanced. In other words, the number of
>>> instances available in one class is far more than the other one.
>>>
>>> Would you please let me know which evaluation measure is the best one to
>>> compare different methods in imbalanced situations? Please note that
>>> predicting the label of instances of the class which contains lower
>>> instances is really harder than predicting the labels of the other
>>> instances and I am looking for a evaluation measure which consider this
>>> issue.
>>>
>>> I am wondering if you also provide me a reference for your opinions.
>>>
>>> Thanks a lot,
>>> Best regards,
>>> Hamed
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Want fast and easy access to all the code in your enterprise? Index and
>>> search up to 200,000 lines of code with a free copy of Black Duck
>>> Code Sight - the same software that powers the world's largest code
>>> search on Ohloh, the Black Duck Open Hub! Try it now.
>>> http://p.sf.net/sfu/bds
>>> _______________________________________________
>>> Scikit-learn-general mailing list
>>> Scikit-learn-general@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Want fast and easy access to all the code in your enterprise? Index and
>> search up to 200,000 lines of code with a free copy of Black Duck
>> Code Sight - the same software that powers the world's largest code
>> search on Ohloh, the Black Duck Open Hub! Try it now.
>> http://p.sf.net/sfu/bds
>> _______________________________________________
>> Scikit-learn-general mailing list
>> Scikit-learn-general@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>
>>
>
>
> ------------------------------------------------------------------------------
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
