I believe one of the original questions was how to discern reputable journals from those that publish dubious or biased results...or do not accomplish proper peer review. I can point to a couple of red flags that can be noticed without too much effort and I have observed:
1) If the articles in the journal come mostly from the same institution in which the editor in chief is located, chances are the buddy system has overwhelmed objectivity...especially if the editor is a co-author in most. 2) If orthographic and syntax errors are widespread, probably the review process was not thorough. 3) If the statistics are grossly inappropriate (for example running an ANOVA with 12 treatments, but only 1 or two replicates per treatment), adequate peer review was clearly not in place. Now these may look like extreme cases, but I have seen too many examples similar to the above to wonder how widespread these cases are. I have even received requests to review papers for certain journals in which I have been asked to be more lenient than if I was reviewing for a major journal. This poses a particular dilemma: Is all science not supposed to be measured by the same standards of quality control regardless of whether the journal is institutional, regional, national or international? I would like to think it should be... Edwin ------------------------------------------------------------------ Dr. Edwin Cruz-Rivera Assist. Prof./Director, Marine Sciences Program Department of Biology Jackson State University JSU Box18540 Jackson, MS 39217 Tel: (601) 979-3461 Fax: (601) 979-5853 Email: [email protected] "It is not the same to hear the devil as it is to see him coming your way" (Puerto Rican proverb) > Martin Meiss said: > This should be more than an exercise in rhetoric; we need > formulations that in simple terms expose the fundamentals of the process, > acknowledge its weaknesses, and distinguish it from phony imitators. > I sure don't have the answers, but I think that we as a community > could come up with them. > > I don't think "could" is strong enough. We as a community MUST come up > with them. > > > > > > >> Mr. Hamazaki's example, whether it is accurate or not, >> illustrates >> one of my points. Just to get by in our professional lives, scientists >> must >> have "faith" in the social institutions, such as peer review, that we >> have >> created. And yet we all know that social institutions are inherently >> corruptible. Not only peer-review, but many other aspects of the >> practice >> of science, are rooted in these corruptible institutions. >> Besides the issue I raised earlier, that of becoming too >> complacent >> in our acceptance of our own perspective, there is the issue raised on >> the >> earlier posts of this thread: How to demonstrate to students (and other >> people who are not scientific professionals) that not all "peer review" >> is >> created equal. Some journalists, in an attempt to be fair-minded and >> objective, think they have to give equal time to holocaust deniers and >> to >> survivors of concentration camps. This same tendency will give equal >> weight >> to "our" and "their" peer-review processes. >> Imagine that you are in a debate on a talk show with an ideologue >> who >> cites dubious research results in a dubious journal, but claims that the >> work is peer-reviewed. What do you say? "That isn't REAL peer-review", >> "Is so!", "Is not!". >> Suppose the show host is smart and stops this and asks how to >> distinguish between valid and invalid peer review. What do you say? >> "We've >> been doing it this way for many years."? "This is the scientific >> consensus >> of how it should be done."? "This is the method used by people who >> think >> right"? Try to come up with a wording that would make sense to a lay >> audience and that couldn't be used by the opponent with equal >> plausibility, >> at least to the ears of the lay people whose taxes are funding your >> research. >> This should be more than an exercise in rhetoric; we need >> formulations that in simple terms expose the fundamentals of the >> process, >> acknowledge its weaknesses, and distinguish it from phony imitators. >> I sure don't have the answers, but I think that we as a community >> could come up with them. >> >> Martin >> >> 2009/7/8 Hamazaki, Hamachan (DFG) <[email protected]> >> >>> In regard to this issue, we should remind that we scientists also fall >>> into >>> this trap. In publishing a paper, we often look for a journal that has >>> high >>> probability of being published. In a way, all you need is several >>> likely >>> minded peers to have your paper published in a peer-reviewed journal. >>> >>> Below is a quote: >>> Hilborn, R. 2006. Faith-based fisheries. Fisheries 31:554-555. >>> >>> A community of belief has arisen whose credo has become >>> “fisheries >>> management has failed, we need to abandon the old approaches and use >>> marine >>> protected areas and ecosystem-based management.” I fear that >>> this >>> belief has >>> shaded the peer review process so badly that almost any paper showing a >>> significant decline in fish abundance or benefits of marine protected >>> areas >>> has a high probability of getting favorable reviews in some journals >>> regardless of the quality of the analysis. Critical peer review has >>> been >>> replaced by faith-based support for ideas and too many scientists have >>> become advocates. An advocate knows the answer and looks for evidence >>> to >>> support it; a scientist asks nature how much support there is for >>> competing >>> hypotheses. >>> >>> http://www.fisheries.org/afs/docs/fisheries/fisheries_3111.pdf >>> >>> >>> Toshihide "Hamachan" Hamazaki, PhD : 濱崎俊秀：浜ちゃん >>> Alaska Department of Fish & Game >>> Division of Commercial Fisheries >>> 333 Raspberry Rd. Anchorage, Alaska 99518 >>> Ph: 907-267-2158 >>> Fax: 907-267-2442 >>> Cell: 907-440-9934 >>> E-mail: [email protected] >>> >> >
