On Fri, May 14, 2010 at 9:20 AM, Vadim Axel <axel.va...@gmail.com> wrote: > Hi guys, > > I apply multi-class major voting scheme for three classes (all pairs > classification). I try to understand how the confusion matrix should look > like when two classes in a pair classification are not discriminated (chance > level). Consider pathological case where classes 1,2 and 2,3 are classified > with 100% and 1,3 are at chance level (50%). The confusion matrix I which > get looks like: > 0.584 0.083 0.333 > 0 1 0 > 0.327 0.071 0.602 > > So, all of sudden it seems that classes 1 and 3 are discriminated. Isn't it > paradoxical? > > When I checked out how I get this result, I have found that it indeed makes > sense. Consider class 1 as a correct label: > pair 1: the classification of classes 1,2 always results in '1' (we are at > 100%, by definition) > pair 2: the classification of classes 1,3 results in half trials in '1' and > other half in '3' (we are at chance by definition). > pair 3: the classification of classes 2,3 results in half trials in '2' and > other in '3' (in case that classes are unrelated, the classifier should be > at chance here). > > The bottom line: since all (1) pairs and half (2) pairs results in '1', I am > already at 50% hit rate for correct class. > > What do you think about all this? Is there any flaw in my logic? > If someone is interested, I can send my matlab simulation.
looks right to me if the tie-breaker is unbiased Probs if 1 is tru: >>> 0.5*0.5 + 0.5*0.5/3. # 3 wins, by majority and tie-breaker 0.33333333333333331 >>> 0.5*0.5/3. # 2 wins, by tie-breaker 0.083333333333333329 >>> 1-(0.5*0.5 + 0.5*0.5/3. + 0.5*0.5/3.) # 1 wins 0.58333333333333337 Josef > > Thanks for help, > Vadim > > > > > > > _______________________________________________ > Pkg-ExpPsy-PyMVPA mailing list > Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org > http://lists.alioth.debian.org/mailman/listinfo/pkg-exppsy-pymvpa > > _______________________________________________ Pkg-ExpPsy-PyMVPA mailing list Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-exppsy-pymvpa