P Nelson wrote: > Interesting read and I like that you included people's comments as that > is very useful in understanding what is happening during the test. > > My only comment is that if you want to test if people can actually tell > the difference between two options is to give them three samples, of > which, two are identical. (I did not read the whole thread so I'd this > was already mentioned, my aplologies.). The problem with pick A or B is > you have a random 50% chance of getting it right. But when you have to > say A=B or A=C or B=C, and then select the one that is different, then > it is very difficult to generate a correct result through guessing. > > It is strange watching people adamantly claim that they can tell the > difference between the three samples, but then they get it wrong of > which of the two out of three are identical! > > Not a slam of what you did, but just a suggestion if you or someone else > does a study in the future. > > Paul
Yup, the classic ABX test. I agree the ABX methodology would be better. The pragmatic issue is whether doing this would be possible "in the wild". If I gave everyone 3 samples and said: here guys; this is Sample A - "it's lossless", now here's Sample B - "it's MP3". Have a listen to A & B, now which is X? It would of course be VERY easy for anyone to compare files to report the answer beyond just listening... In a test like that, outside of an actual lab where conditions can be controlled or you have full faith that the person testing would not do anything beyond just playback, it would be impossible to distinguish who may have "cheated". A more sophisticated test would be an actual software program which can do the testing randomly and submit the result remotely to a central repository. This would require someone to write some fancy software and the respondents to run it on their compatible machine... But it won't be "naturalistic" anymore in that you now cannot run it on JPlay/Amarra/Decibel/JRMC/Touch... which many claim sounds better than the standard playback systems of their computer. Ultimately, the results showed that MP3 didn't sound too bad after all... I'm still waiting for Neil Young to respond :-) ------------------------------------------------------------------------ Archimago's Profile: http://forums.slimdevices.com/member.php?userid=2207 View this thread: http://forums.slimdevices.com/showthread.php?t=98010 _______________________________________________ audiophiles mailing list [email protected] http://lists.slimdevices.com/mailman/listinfo/audiophiles
