Is there any particular reason why high quality peer review cannot be crowd sourced or self-organized?
It seems that most circles, societies and recognitions of learned people and experts are in effect self-organized, in that those who are deemed expert are deemed such by other experts. So a self-organizing peer review system based on collaborative wiki-like principles seems conceptually possible. In the context of a wiki, I could imagine a review system something like this: 1. Paper is published in "Draft:" space, where anyone can contribute, critique, collaborate (if the owner wishes), discuss with the author, or suggest observations on its "Draft talk:" page. 2. When the author/s feel ready for formal review (or other criteria of the community) the paper is "frozen" and reviewers are decided. Perhaps there will be a page where author/s inform the community they want to move a paper to review, and members of the community nominate themselves or otherwise agree who the formal reviewers will be. As with normal peer review there may be reviewers with specialist skills or expertise, or a balance of reviewers, and some papers will surely attract more scrutiny than others. The merits of the paper will surely influience and be judged by the caliber of its reviewers. 3. A key change would be that reviewers' identities would be public. Although this would remove the usual complete separation of author and reviewer, it also means that the reviews, the relationships, and the approach will be completely public and itself open to scrutiny for all future time. For those whose repuytation and career rest on clearly ethical behavior in their academic work, this might be if anything at least as powerful an incentive to review within community guidelines. Future emergence of any untoward behavior, or any strange attitudes or unexpected review posts at review will be picked up on, and this total transparency has the potential to be as effective an encouragement of highest standards and deterrent of ethical breach as any formal separation. 4. When reviewers are agreed, the paper is moved to "Review:" space. The old talk page (now "Review talk:") remains open for general participation, and a new page "Review discussion:" is opened for formal review. 5. The paper is reviewed, and if needed edited. If so decided fresh reviewers may be needed at some point. When review and editing seem complete and no further edits or responses by reviewerrs are likely, the community considers whether the paper should be moved to the "Paper:" namespace which is spidered and treated as published by the community. Disclaimer, I'm not an academic so I have no idea whether exactly this model is already in use somewhere. If so, what are its shortfalls and how does it work in practice? FT2 On Tue, May 22, 2012 at 11:28 PM, Benj. Mako Hill <[email protected]> wrote: > > If the result of that turmoil is that tax-payers get access to the > research they funded and that the product of scientific knowledge is > increasing accessible without pay-walls and fees that create > roadblocks and exacerbate existing inequalities, I am willing to put > up with a little turmoil. > > There are already thriving open access journals in many fields and > plenty of reasons to believe that peer review is not at risk. But even > if we had no idea *how* things would work out, let's not let a lack of > imagination keep us from standing up for something that is > right. Access to scientific knowledge is deeply important -- and a key > part of why Wikipedia exists today. Just as Wikipedia has does in the > area of reference works -- and lets not pretend Wikipedia growth has > been without turmoil -- let's *make* it possible. > > Regards, > Mako > > > -- > Benjamin Mako Hill > [email protected] > http://mako.cc/ > > Creativity can be a social contribution, but only in so far > as society is free to use the results. --GNU Manifesto > > _______________________________________________ > Wiki-research-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > >
_______________________________________________ Wiki-research-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
