Mr. Hamazaki's example, whether it is accurate or not, illustrates
one of my points. Just to get by in our professional lives, scientists must
have "faith" in the social institutions, such as peer review, that we have
created. And yet we all know that social institutions are inherently
corruptible. Not only peer-review, but many other aspects of the practice
of science, are rooted in these corruptible institutions.
Besides the issue I raised earlier, that of becoming too complacent
in our acceptance of our own perspective, there is the issue raised on the
earlier posts of this thread: How to demonstrate to students (and other
people who are not scientific professionals) that not all "peer review" is
created equal. Some journalists, in an attempt to be fair-minded and
objective, think they have to give equal time to holocaust deniers and to
survivors of concentration camps. This same tendency will give equal weight
to "our" and "their" peer-review processes.
Imagine that you are in a debate on a talk show with an ideologue who
cites dubious research results in a dubious journal, but claims that the
work is peer-reviewed. What do you say? "That isn't REAL peer-review",
"Is so!", "Is not!".
Suppose the show host is smart and stops this and asks how to
distinguish between valid and invalid peer review. What do you say? "We've
been doing it this way for many years."? "This is the scientific consensus
of how it should be done."? "This is the method used by people who think
right"? Try to come up with a wording that would make sense to a lay
audience and that couldn't be used by the opponent with equal plausibility,
at least to the ears of the lay people whose taxes are funding your
research.
This should be more than an exercise in rhetoric; we need
formulations that in simple terms expose the fundamentals of the process,
acknowledge its weaknesses, and distinguish it from phony imitators.
I sure don't have the answers, but I think that we as a community
could come up with them.
Martin
2009/7/8 Hamazaki, Hamachan (DFG) <toshihide.hamaz...@alaska.gov>
> In regard to this issue, we should remind that we scientists also fall into
> this trap. In publishing a paper, we often look for a journal that has high
> probability of being published. In a way, all you need is several likely
> minded peers to have your paper published in a peer-reviewed journal.
>
> Below is a quote:
> Hilborn, R. 2006. Faith-based fisheries. Fisheries 31:554-555.
>
> A community of belief has arisen whose credo has become “fisheries
> management has failed, we need to abandon the old approaches and use marine
> protected areas and ecosystem-based management.” I fear that this belief has
> shaded the peer review process so badly that almost any paper showing a
> significant decline in fish abundance or benefits of marine protected areas
> has a high probability of getting favorable reviews in some journals
> regardless of the quality of the analysis. Critical peer review has been
> replaced by faith-based support for ideas and too many scientists have
> become advocates. An advocate knows the answer and looks for evidence to
> support it; a scientist asks nature how much support there is for competing
> hypotheses.
>
> http://www.fisheries.org/afs/docs/fisheries/fisheries_3111.pdf
>
>
> Toshihide "Hamachan" Hamazaki, PhD : 濱崎俊秀：浜ちゃん
> Alaska Department of Fish & Game
> Division of Commercial Fisheries
> 333 Raspberry Rd. Anchorage, Alaska 99518
> Ph: 907-267-2158
> Fax: 907-267-2442
> Cell: 907-440-9934
> E-mail: toshihide.hamaz...@alaska.gov
>
