Very interesting results. Thank you for the time and care you spent doing this.
I am not a researcher, just an amateur like yourself :), but I have a few methodological concerns: 1) You bundled the files into two complete bundles (all mp3s were in group A, all lossless in B). In doing so it becomes much more difficult to draw any conclusions about the detectability/preference of mp3 encoding as it pertains to any one pair of files. It also allows people to focus intently only on the files they care to listen to, or those which they came across first, and then to make conclusions about the whole group. Additionally, it prevents you from changing the order of the files (selection of group A by most respondents may have been primacy effect, to some degree). Asking about confidence on a per-file basis as you did would seem to mitigate some of these concerns but I don't think it fully addresses the potential problems. 2) There's no control group. This could have helped identify any possible the primacy effect, too (if you'd had each individual file pairing done independently it would have been possible to have two identical files as one of the pairs). 3) Your main question "which set sounded inferior" had 3 answers: A, B, or "no audible difference". This, paired with the question about confidence does a decent job of answering the question "which sounds better" but I wonder if it does a good enough job of answering the question "is mp3 distinguishable from lossless". There could be a subset of people who had a hard time developing an opinion on which one sounded inferior, but an easy time distinguishing between the files. These people would not have wanted to answer "no audible difference" so may have taken a guess for the question "which set sounded inferior", but for "how difficult was it to come to your conclusion" they might have said "easy" (since it was easy for them to distinguish the files). Maybe I'm splitting hairs here; these are just some thoughts that came to mind while reading the results. 4) I also wonder about the decision to use such an unorthodox mp3 encoding technique. I understand your rationale for doing so, but in the end it seems that your conclusion necessarily becomes "people tend to prefer this unusual method of audio processing over the original files". If you'd used a more typical encoding method then your conclusions could be more useful by applying more broadly to mp3s in the "real" world. These all sound like harsh criticisms. They're not. Your survey is, to me, exactly the kind of stuff audio publications should be doing. You clearly put a lot of though into this; it was a great read! ------------------------------------------------------------------------ azinck3's Profile: http://forums.slimdevices.com/member.php?userid=3967 View this thread: http://forums.slimdevices.com/showthread.php?t=98010 _______________________________________________ audiophiles mailing list audiophiles@lists.slimdevices.com http://lists.slimdevices.com/mailman/listinfo/audiophiles
