Hi,
In addition to what has already been suggested, especially Chi^2 and MCC, I
would suggest this:
http://dx.doi.org/10.1109/PRNI.2012.14 (full disclosure: it is one of
my papers)
which is, in short, a Bayesian equivalent of Chi^2 / MCC, which works for binary
and multi-class and does not suffer most (if not all) the problems of Chi^2 and
MCC. Notice that, in the multi-class case, the proposed method is also extended
to detect the case where only some of the classes are not discriminated, but not
all.
A Python implementation of the proposed algorithm (a bit updated since that
paper) is here:
https://github.com/emanuele/inference_with_classifiers
Feel free to ask if you need help.
An extended version of the paper is in preparation.
Best,
Emanuele
On 07/22/2014 05:26 PM, Hamed Zamani wrote:
Hi,
I am working on a binary classification problem in which both training and
test data are highly imbalanced. In other words, the number of instances
available in one class is far more than the other one.
Would you please let me know which evaluation measure is the best one to
compare different methods in imbalanced situations? Please note that
predicting the label of instances of the class which contains lower instances
is really harder than predicting the labels of the other instances and I am
looking for a evaluation measure which consider this issue.
I am wondering if you also provide me a reference for your opinions.
Thanks a lot,
Best regards,
Hamed
------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
