We recently carried out an online survey of current peer review practice. We got 200 replies, representing many more than 200 journals (some respondents were multi-journal editors, or publishers). You can find the results at www.alpsp.org/pub4.htm
Sally Sally Morris, Secretary-General Association of Learned and Professional Society Publishers South House, The Street, Clapham, Worthing, West Sussex BN13 3UU, UK Phone: 01903 871686 Fax: 01903 871457 E-mail: [email protected] ALPSP Website http://www.alpsp.org Learned Publishing is now online, free of charge, at www.learned-publishing.org ----- Original Message ----- From: "Stevan Harnad" <[email protected]> To: <[email protected]> Sent: 26 January 2001 14:05 Subject: Re: ePrint Repositories [+ Peer Review] > On Fri, 26 Jan 2001, Simon Buckingham Shum wrote: > > > I disagree with your terminology. Ask any academic what > > "classical peer review" is, and they will tell you about the > > "anonymous reviewer, no right of reply for author" process. > > The one to ask is not "any academic" but journal editors. You will find that > most journals offer referee anonymity as an option (which can be waived by the > referee, and often is). And most competent editors will mediate, even for > several rounds, if there is a substantive basis for interaction between authors > and referees. > > It is only to some (disappointed) authors that the system sometimes > looks rigid, conspiratorial and biassed: In reality, editors would like > to publish the best papers, and will do whatever it takes to make sure > they do, including following up author/referee disagreements where it > looks as if it is warranted. > > If you remove the option of referee anonymity you invite more bias than > you remedy: How is one to do an honest negative review (if that is what > is called for) of someone more senior than oneself, or someone who may > be refereeing one's next grant proposal? The short answer to why > referees need the anonymous option is: for exactly the same reason we > vote anonymously in a democracy: to make sure we can exercise our > choice without fear or favor. Sure, this can be abused in the other > direction too, but that's what a competent editor is there for: the > reviewer is never anonymous to the editor. > > > Our conversational model places author-reviewer interaction at the > > centre right from the start, makes this 'intellectual trace' > > accessible to wider peers, and preserves an edited, open version of > > it with the publication. > > I misunderstood about referee anonymity then; I thought anonymity was > possible within the "closed" phase of author/referee interaction, > mediated by the editor, in your system: If referee anonymity is not > possible, then yours is not classical peer review. If the interactions > are, in addition, visible not only to the author/editors/referees but > to the wider community as well, then this your system is still further > removed from classical peer review, and needs empirical testing to > determine the resultant quality it delivers before it can be declared > ready for adoption. > > Note that the archiving of the successive embryological phases of > preprint/revision/postprint provides the "intellectual trace" you > desire, even with classical peer review (plus self-archiving of all > those embryological stages, if the author wishes), but it leaves that, > rightly, voluntary. The same is true with the archiving of referee > reports, comments, responses, and postpublication revisions: These too > are possible in the classical system, but voluntary. > > As soon as you make variants that could have substantive effects on quality > (e.g., no anonymity option, public review) mandatory, you have to test and > demonstrate what effects that has on quality; till then, everyone is best > advised to stick with tried-and-tested classical peer review. > > > BBS has implemented commentary and > > author response as best it could in the paper medium, but as Stevan > > himself has argued, the net enables a more rapid tempo for scholarly > > discourse. This is simply not practical in paper. > > Correct, and that is why Psycoloquy was founded in 1989. > > > CogPrints restricts this benefit to "pre-journal" status > > contributions. > > CogPrints is not a journal but an archive. Any embryological phase can > be self-archived therein, be it pre-refereeing preprint, revised draft, > refereed final draft, postpublication update, referee report, > commentary, or response -- as long as the author of each type of item > chooses to self-archive it. > > > JIME gives it > > first class status as a mode of discourse suitable for journal > > submissions, particularly in a new interdisciplinary context. The > > context of use (disciplinary factors and user expertise may be > > variables that determine success of more radical models). > > But we will need a few years to find out exactly what the quantity and > quality of the papers undergoing any new, untested system like this > will actually turn out to be. Till then, no conclusions can be drawn, > and certainly no new system should be adopted. > > > We are certainly agreed that there are no proven substitutes for > > organised peer review. (Future technologies may appear to provide > > automatic analyses of quality, but you can't beat humans.) To make > > review happen, you need to engage reviewers in a social contract. To > > make this happen requires some form of coordination, up until now the > > role of the journal. > > > > However, it is not inconceivable that the net will enable > > decentralised coordination of some sort, which is able to create the > > social contract to review (or provide some other kind of incentive?), > > without any notion of a Journal or other Learned Body to establish > > the contract. Sort of Napster peer review model. > > Not inconceivable, but until demonstrated, this is merely untested > speculation. > > > But what's the > > payback for the reviewer? Short of paying reviewers, I don't know of > > anyone who's successfully devised other motivations yet. > > Paying reviewers is yet another untested and potentially corruptive > variant on classical peer review. > > -------------------------------------------------------------------- > Stevan Harnad [email protected] > Professor of Cognitive Science [email protected] > Department of Electronics and phone: +44 23-80 592-582 > Computer Science fax: +44 23-80 592-865 > University of Southampton http://www.ecs.soton.ac.uk/~harnad/ > Highfield, Southampton http://www.princeton.edu/~harnad/ > SO17 1BJ UNITED KINGDOM > > NOTE: A complete archive of the ongoing discussion of providing free > access to the refereed journal literature online is available at the > American Scientist September Forum (98 & 99 & 00 & 01): > > > http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html > > You may join the list at the site above. > > Discussion can be posted to: > > [email protected]
