You can't punish people into refereeing more. I suggest you reward
them. I've been meaning to publish a paper on this but feel free to
steal this, if UI haven't stolen this from someone else.
We have citation impacts for published papers and authors (see
Publish or Perish). Why not refereeingimpacts for individuals? Each
journal would annually publish the names and numbers of times each
reviewer refereed. Since referees are the gatekeepers, the number of
reviews summed across all journals would yield a number indicating
how important the person is in deciding what gets published. But this
can be gamed, as someone can referee lots of small notes, so a second
index might correct by journal impact factors. A few reviews of
papers in high citation journals would count for a lot more than many
in low impact journals.
Then you can correct for time, folks who don't actively review would
have lower reviewer impacts than those who continue to do so. Take
the indices in the program "Publish or Perish" and convert them.
Finally, journals could be re-ranked by the reviewer impact ranks. Do
they have heavy duty reviewers or not (I am not sure what this would
do, but it might put pressure on journals to get senior people to
review. It might freeze out junior folks which could be an undesired
consequence).
Hiring and tenure decisions could then look not just at publications
but at what editors thought of individuals. Are they people editors
think of as at the cutting edge and are they cooperative (important
for hires)? The shortage of reviewers might disappear. Reviewing
would be recognized and would receive credit.
This would be easier than creating a credit system with a high
potential for mischief.
David
At 04:10 AM 7/23/2010, Jeremy Fox wrote:
David,
I had hoped it was implicit in our article that journal participation in the
PubCred system is voluntary. It is explicit in the article that the more
journals in the field that participate, the better. Otherwise, authors who
don't wish to review can avoid the system by submitting to non-participating
journals.
As stated in the article, the centralized PubCred Bank online system keeps
track of credits. As stated in the article, discussion is needed about who
would pay for, operate, and improve the online system. We suggested a
non-profit consortium supported by online ads, but it's unclear if this is
workable.
Many journals already conduct their business in English, which already
'discriminates' against those whose first language is not English (though
'discriminates' is not the word I would choose). I am unclear how the
proposed PubCred system would impose further discrimination.
I am afraid it is not obvious to me why only journals of a certain size
would benefit from the PubCred system; can you explain? Note that a journal
that does not draw its authors and reviewers from the same 'pool' as the
journals participating in the PubCred system would have no reason to join
the system (indeed, would have good reason not to join). The PubCred system
works best if it includes all and only the journals to which a very large
set of authors and referees would consider submitting to and reviewing for.
It is of course impossible to satisfy this criterion perfectly, unless every
journal in all of science (and the social sciences and humanities!) is
included. But I believe this criterion can be satisfied well enough for the
system to work for the participating journals. I am unclear how journals
that have no reason to participate and choose not to participate would be
harmed; can you explain?
Regarding Chinese names and common English names such as David Duffy, please
see previous comment. Further, all current online ms handling systems
already assign each person a unique login name, such as their email address.
The PubCred Bank would presumably do the same.
In summary, I agree that the devil is in the details. That's why Owen and I
are working very hard through various avenues, including Ecolog-L, to
solicit detailed, constructive feedback. I'm currently spending several
hours per day publicizing PubCreds, soliciting feedback, and thinking about
and responding to feedback. I do so not because I want to see PubCreds
imposed on the community, but because I believe the 'tragedy of the peer
review commons' is a serious and growing problem and I want to see an
effective solution (whether PubCreds or something else) agreed upon by the
community.
On Thu, 22 Jul 2010 13:04:32 -1000, David Duffy <[email protected]> wrote:
>For the the PubCred system
>
>Which journals will be asked (or allowed?) to participate? Who decides?
>
>Who will keep track of the credits?
>
>Many editors in smaller journals already have enough to do and may
>not be overly enthused, how would you deal with the extra burden on them?
>
>Won't this discriminate horribly against those for whom English is
>not a first language?
>
>How will you tell Chinese names apart as their 7,000 characters lose
>something when translated into our alphabet? Or how will you tell the
>various David Duffys apart?
>
>The devil is in the details.
>
>Cheers,
>
>David Duffy
>
>
>At 10:38 AM 7/22/2010, Jeremy Fox wrote:
>>The most common concern raised about the proposed system is how
to allow for
>>people (especially students and postdocs) who aren't asked to review very
>>much, and so lack PubCreds through no fault of their own.
>>
>>Our article notes this issue and suggests 'overdrafts' as a means to deal
>>with it. But many other solutions have been suggested, and there
are aspects
>>of this issue which Owen and I did not have the space to address in our
>>article. So I've just posted a blog entry on the PubCred website further
>>discussing this important issue:
>>
>>http://www.ipetitions.com/petition/fix-peer-review/blog/2594
>>
>>I summarize many of the alternative suggestions we've received so
far on how
>>to address this issue, and highlight the importance of a period of data
>>collection before the PubCred system 'goes live'. I also note that there's
>>often no clear distinction between those who lack PubCreds
'through no fault
>>of their own', and those who have 'only themselves to blame' for their lack
>>of PubCreds. Like any 'bank balance', a PubCred balance reflects both
>>'income' and 'expenses'. Just because someone has little 'income' doesn't
>>mean they've 'spent' wisely, which makes it a little tricky to choose the
>>best way of helping 'low income' people like students and postdocs.
>
>
>
>
>David Cameron Duffy
>Professor of Botany and Unit Leader
>Pacific Cooperative Studies Unit (PCSU)
>University of Hawai`i
>3190 Maile Way St. John 410
>Honolulu, HI 96822-2279
>(808) 956-8218 phone
>(808) 956-4710 fax / (808) 956-3923 (backup fax)
>email address: [email protected]
>=========================================================================
David Cameron Duffy
Professor of Botany and Unit Leader
Pacific Cooperative Studies Unit (PCSU)
University of Hawai`i
3190 Maile Way St. John 410
Honolulu, HI 96822-2279
(808) 956-8218 phone
(808) 956-4710 fax / (808) 956-3923 (backup fax)
email address: [email protected]
