On Thu, 16 Jan 2003, Andrew Odlyzko wrote [in part]: [ao]> The recent postings to this list about rejection [ao]> rates and costs of peer review point out yet [ao]> another way that costs can be lowered: Elimination [ao]> of the wasteful duplication in the peer review system.
Publishers of several journals can achieve economies of scale by using the same staff to oversee multiple journals. Economies of scale for the peer reviewers would require centralized peer review for a particular field or discipline. This approach has been tested in Canada by the Canadian Breast Cancer Research Initiative (CBCRI): <http://www.breast.cancer.ca/>. The Canadian Institutes of Health Research (CIHR), the National Cancer Institute of Canada (NCIC), the Canadian Cancer Society, Health Canada, and the Canadian Breast Cancer Research Foundation (CBCF), all use the same peer review system (that of the NCIC) for the evaluation of research proposals submitted directly to the CBCRI. However, this hasn't really achieved much economy of scale, because some of these agencies (NCIC, CIHR, CBCF) also, for what I think are good reasons, also peer-review those breast-cancer applications that are sent directly to them, rather than to the CBCRI. The individual research teams make the decision (and some choose, again for what I think are good reasons) to submit essentially the same application to more than one of these various agencies. Different peer-review committees judge "quality" according to somewhat different criteria, and involve committee members who may be true "peers" in relation to one aspect of as research field or discipline, but not in relation to another. The mix of expertise matters. So, many research teams prefer to have an opportunity to "take more than one kick at the can". If peer-review is regarded as a process of weighted randomization, then, from the point of view of an individual research team, the probability of successfully obtaining support is increased if multiple applications are submitted. The situation isn't very different for peer-review of research reports, except that the number of "peers" involved in the review process is usually much smaller (e.g. 2 or 3 people, instead of about 10). The smaller the number of reviewers, the greater the variance in the score or rating of perceived quality. Jim Till University of Toronto
