Greetings :-
Professor Langseth argues that the values of "professionalism" and
"review quality" are better served by a double-blind review procedure
(neither reviewers nor authors are told one another's identities, reviewers
are not told each other's identities) than by a single-blind procedure
(reviewers are told authors' identities, but not vice versa, and reviewers
are not told one another's identities).
A third alternative, open review (common knowledge of all identities
among all immediate stakeholders, all of whom read all of the reviews) is
practiced by _Behavioral and Brain Sciences_, unless an individual reviewer
requests anonymity (in which case that reviewer's identity is not told to the
other participants).
Of course, the fruit of the BBS process is _public_ peer review (with
all names and comments disclosed, and with author rebuttal). But any
scientific publication is fair game for public scrutiny once it is in print.
BBS' editor (about to step down after long service), is the distinguished
Stevan Harnad, a productive scientist and innovator in the effective
dissemination of scientific information.
Obviously, plausible arguments can be mustered in favor of and in
opposition to all three of these methods. Open review is interesting, I
think, since it proceeds from the premise that the nub of the problem is
accountability. That scientific debate is otherwise characterized by its
"openness" offers another thread of argument in favor of open review.
With respect to double-blind review, I think it is reasonable to ask
whether anonymity can in fact be achieved. Typically, the paper is available
on the Internet at the author's website and in many cases, reviewers have
been selected according to criteria which tend to ensure that they would know
who is working on what problem by what methods. Conversely, under all three
protocols, referees write a prose narrative to explain their verdict, and
many people have "signature" writing styles. (Fortunately, my own writing
style is hard to spot, much as Philippe Smets' is.)
I think it is clear that "anonymity failure" would be especially likely
in a small community. If that community's members are also adepts at
inference under uncertainty, then unless they are deluded, "failure"
predictably collapses into "sham." Even in a large community where median
inferential prowess is doubly uncertain, like the AAAI/IJCAI general
population which lately practices double-blind reviewing, the collapse into
"sham" can be only too evident.
Best regards to all.
Paul Snow
