--On Wednesday, November 14, 2001, 1:22 PM -0800 Smets Philippe <[EMAIL PROTECTED]> wrote:
> At Queen Victoria time, the British Parliament passed a law making > illegal the homosexuality between male partners. Why only male, and > not female? Because making illegal the homosexuality between females > partners requires first to acknowledge that such a thing could exist, > an idea totally unacceptable at that time. > > To establish a double blind review system requires first to > acknowledge that reviewers might be dishonest. Is that acceptable? Actually, I believe that we are facing a task that is easier than that faced by the British Parliament at Queen Victoria's time. There is a substantial body of literature in the field of experimental design that testifies to the existence of so-called "subject experimenter artifacts." These refer to (1) subjects' behavior that leads to biased results, (2) biases in experimenter's perception of results of empirical studies, and (3) interactions between the subjects and the experimenters. These effects are so strong that it is a common knowledge that results of empirical studies should be trusted unless they have been based on some kind of blinded design. We know of subjects being affected by the treatment (placebo effect). An acceptable remedy for that is blinding the subjects, so that a subject does not know whether he/she/it is being treated or not. A similar thing happens at the experimenter side, when the experimenter makes unconscious errors in reading and interpretation of data (so, no malicious intent!) when he/she knows what he/she is looking for. The remedy for this is double-blinding so that neither the subject nor the experimenter know which treatment is being applied or which data is being collected or looked at. In a hospital setting, a nurse is asked to administer a drug and he/she does not know which of the pills that he/she is giving to a patient is medicine and which is placebo. A consciencious physicist may purposefully decalibrate an instrument, take readings that are meanigless to him/her, and then, at the very end of the experiment, when all the data have been collected and only need to be plugged into a formula and analyzed, transform the collected data into true, meaningful measurements. A spectacular example that involves the second and the third effect is due to, I believe, Rosenthal (I don't have the reference handy). It describes a study that tests subject-experimenter effects in empirical studies. Two groups of students (sections of the same class?) were asked to teach rats to find food in a maze and measure the learning times. The first group was told that the rats belonged to a special breed of "maze-smart" rats. The second group was told that the rats they received were just ordinary stupid laboratory rats. There was an amazingly large difference in the results -- the "maze smart" rats learned much faster than the "ordinary rats" and, as you have probably guessed, both groups of rats came from the same breed. How does this relate to double-blinding in the UAI reviewing process? If merely framing a problem can induce a different behavior in humans and if even rats can respond differently to treatment, what can we expect from a problem as complex as reviewing a scientific paper? While, as some of my predecessors noted, it is a matter of balance between the amount of work on the part of the authors (hiding their identity) and the program chairs (organizing the double-blinding reviews) and the resulting improvement in the quality of the reviews and the conference program, there is no doubt to me that double-blinding should improve the process. There is no need to refer to dishonesty. Even when we try to be honest, we may be "dishonest" enough :-). Marek -------------------------------------------------------------------------- Marek J. Druzdzel http://www.pitt.edu/~druzdzel
