I experience this effect all the time. So much so that I now make
allowances for it - e.g, I refuse to listen to MP3s and FLACs/CDs side
by side because I *do not want* to learn how to identify MP3 artefacts
and thereby ruin my enjoyment of all MP3s.
Of course pairwise comparisons produce a sorting; in fact, it's
fairly difficult to sort in any other way. (is the definition of a
connaisseur someone who has enough valuation experience to use non-
comparison sorting algorithms?)
Upon cursory googling, it appears that students are trained to rank
cattle based on small 4-animal classes, where they must justify each
comparison. For amateurs of other domains (codecs, teas, etc.) it
might be interesting to follow the livestock format which includes
grants as well as criticisms: (N beats N+1 because of important
qualities X,Y; although N+1 is superior to N in lesser qualities A,B,
it's still lower placed because of Z*)
In practice, horse show classes tend to be relatively large, and my
understanding is that the judges use a bucket sort variant: they very
quickly stack the class into rough equivalences (eg with a 1-10
scale; cf supra) then go pairwise from there, trying to reserve a
relatively large fraction of the exhibition time for discriminating
between close pairs.
-Dave
* an advanced version of this exercise includes criticism of the best
ranked individual against the (breed/discipline/industry) ideal. As
it's far too easy to be critical (a little knowledge suffices to find
fault with anything concrete), i'd suggest that the true amateur is
capable of appreciating much that is (tautologically!) imperfect.
