Just a couple of brief responses to your comments. Beyond this, I do not intend to continue. We do have substantially different visions of the future of peer review, and they have not changed much since we first started corresponding back in 1993.
> ...if your prediction happens to be wrong, making the prediction > anyway will have a negative, retardant effect on self-archiving and open > access. (Of course, if you are right, then concerns about these changes > will still have a negative, retardant effect on self-archiving.) I am not sure it will be negative. It might encourage the transformation by showing the path to a future in which not only information dissemination, but also peer review, are improved. >ao> How does this differ from somebody a decade or two ago that >ao> might have promised that electronic publishing would simply mean >ao> that journals would now be available online, but there would >ao> be no disturbing innovations such as scholars being confused by >ao> uncontrolled preprint distribution? > > I can't see the point (and I'm not sure what you mean by scholars being > confused by uncontrolled preprint distribution!). Some of the opposition to electronic publishing was based on concerns about uncontrolled proliferation in available material. > (Nor does it seem to me that you are making > these predictions because you are recommending that people think twice > about the transition, or first take some remedial measures.) Certainly not. I hope to hasten the transition, by pointing out that it is likely to improve the peer review system. > Self-archiving was tried extensively, and demonstrated to work. Now we > can confidently say it works and recommend it to everyone. Alternatives > to peer review have not been tried or demonstrated to work. Nothing > prevents people from trying to implement controlled experiments on > alternatives to peer review. But until they are done, and the outcome > known, there is no basis whatsoever for linking them to self-archiving > and open-access. Sorry, but alternatives to classical peer review have been tried, and are constantly being tried (quite successfully, in my opinion). That is largely what my article "The fast evolution of scholarly communication" was about. > The fact, though, is that no one KNOWS that your prediction is true. So > dilating on it now -- when its truth cannot even be known, and when, > on the face of it, proclaiming it will merely reinforce people's fears and > hesitations about self-archiving -- can hardly serve a useful purpose. (If > I were you, and I could not in good conscience deny my belief in the > causal connection between self-archiving, open-access, and the changes in > peer review that you described, I simply would not express my belief at > all, rather than risk voicing a fallible belief that is almost certainly > going to have a negative effect on something I regard as very positive, > but also certain.) Well, yes, we don't know whether my prediction is true, but there is evidence for it, for example in "The fast evolution ...." Peer review is undergoing change right now, in front of our eyes, even though few are paying attention. To pretend that nothing will change seems really short-sighted. > ...the peer review generates a reliable, > recognizable, quality-level-tag, a recognizable milestone with an > established track record, along the continuum, on which the would-be user > can depend. Without that, it is not at all clear where a particular paper > stands, in quality and usability, along its own continuum... But even with classical peer review "it is not at all clear where a particular paper stands ..." We get a very weak quality and usability signal from classical peer review, and my contention is that we can obtain many other signals that collectively, if not individually, can be even more useful. > Reinvented or reshaped where, and by whom? As we speak, whether a > self-archiver or not, not a single author of the annual 2,000,000 papers > that appear in any of the hierarchy of 20,000 peer-reviewed journals > published across all disciplines and around the world has stopped > submitting his papers to those journals. Your predictions are merely > speculations. They have not been implemented and tested, and what the > outcome would be if they were tested is not known. I would have to go back and dig up some old messages, but I believe there are several very reputable scholars who have stopped publishing in traditional journals. Furthermore, some estimates have been made of arXiv submissions, and a noticeable fraction of them do not get submitted to journals. (I can testify to some really outstanding papers in mathematics that are available only through arXiv, because their authors simply never bothered to submit them to journals.) Part of our difference is probably rooted in our varying professional experiences. As just one example (there are others in "Tragic loss ...") some of the most interesting developments in mathematics over the last half century were in algebraic geometry (leading to several Fields Medals, the mathematics equivalent of a Nobel Prize). Much of that work was based on several thousand pages published in the un-refereed Springer Lecture Notes in Mathematics (the famous SGA volumes). Why was the field willing to rely on those? Well, it is a long story, but there were enough checks around (such as many people, including graduate student seminars going over those volumes line by line) to convince experts to rely on those papers. It was examples like these, definitely involving review by peers, but not classical peer review, that helped convince me that scholars could thrive under a variety of systems. I spend an inordinate amount of time on this subject in "Tragic loss ..." > Peer review is not a gold standard, but I'm sure you will agree that > any alternative would have to ensure at least the same standard, if > not better: Do you think there is this evidence for the promise you are > holding out above? Yes, I do. Examples such as that of algebraic geometry (mentioned in my previous response above) showed me early on that there is nothing sacred about classical peer review. The law review system is yet another example. > I honestly can't see how you imagine this scaling to the annual > 2,000,000 papers that currently appear in the classically peer reviewed > journals! Absent the peer reviewed journal, how can I know that a paper > has been "vetted by experts of a top caliber"? What tells me that (as > the journal-name currently does) for those 2,000,000 annual papers? And > what now gets the right-calibre experts vetting the right papers (as > editors formerly did, when they invited them to referee?). Do experts > voluntarily spend their precious time trawling the continuum of raw > papers on the net on their own? I do not have the time to respond to all these points, but yes, many experts do "voluntarily spend their precious time trawling the continuum of raw papers on the net on their own." I do know many people whose day starts with a scan of the latest arXiv submissions. Moreover, some put a lot of effort into making what they find more easily digestible for others. (John Baez and his wonderful "This week's finds in mathematical physics" comes to mind.) I am sorry, Stevan, but you are ignoring some of the most interesting evolutionary developments in scholarly publishing in your blind faith that classical peer review is the only thing that stands between us and chaos. > It is up to you, but I do not understand why your conscience tells you > you need to share your speculations (especially when they risk alienating > the majority who are still leery about self-archiving!). Well, I was invited to contribute my thoughts on peer review to a book devoted to the subject, so I did the best I could within the allotted limits. Open archives are not a religion to me, just a step towards a better scholarly communication system, which will also require changes in peer review. Andrew Odlyzko
