dear pymvpa users,
i have predictions from a 3-way classification and a 2-way
classification that i would like to compare with one another. how could
i do this?
1) i could subtract the chance level from each accuracy score, i.e.,
subtract 1/3 from the 3-way classification accuracy and 1/2 from 2-way
classification. not ideal because percentage changes above chance are
not directly comparable anymore. but the approach is pretty intuitive
and permutation inference against chance levels would still be valid.
2) use a different performance metric like (adjusted) mutual information
maybe? methodologically more appropriate probably but maybe confusing
for the readers.
3) but perhaps there are even better ways to do this. for example
examine the 3-by-3 and 2-by-2 confusion matrices and compare
main-diagonal with off-diagonal entries?
any other ideas?
thank you,
michael
_______________________________________________
Pkg-ExpPsy-PyMVPA mailing list
Pkg-ExpPsy-PyMVPA@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
