I have tried to stay out of this debate, for it is clearly futile. But I can stand it no longer. To Annette and others who are attempting to insert the IRBs into methodological issue surrounding research: What methodological expertise do members of IRBs have, *by virtue of being holders of that office*, that other members of the faculty do not have? Are they selected for their particular expertise in methodology? Are they required to have a particularly wide range of experience in that area? Is there some reason to believe that they know more, in general, than the people who are submitting the studies. The answer to all of these questions is, obviously, "no." They volunteer. Or they are "voluntold" by Dept. Heads and Deans. Or they are "elected" by their peers in meaningless exercises that typically descend into "acclamation" because no one else wants to do it. They may have no knowledge of you research area at all; no knowledge even of your wider discipline. As a result, all of your worries about students being "abused" (please!) by having their time wasted in badly-designed studies to the side, there is no reason to believe that what IRBs say about methodology is particularly pertinent, or even that it improves studies more often than it harms them.
I have little doubt that Annette ran up against numerous poorly-designed studies when she was a member of her local IRB. I have no reason to doubt that she, personally, could give people good advice on how to design their studies better. But to argue that the methodological advice that IRBs give should, across the board, have the weight of institutional mandate is, well, simply beyond belief. If you want to have a body that oversees the methodological rigor of the studies that your faculty carry out, then you should, at least (all questions of collegiality and academic freedom to the side, for the moment), have a board that has some claim to expertise in this area. NOT putting all questions of collegiality and academic freedom aside, we should leave it to experts in the specific fields of study (journal editors, conference program chairs, etc.) to decide whether people have methodological competence in their areas of study. It strikes me as quite telling that NO ONE would accept a tenure & promotion committee that didn't call for letters from outside experts, because everyone knows that the members of the tenure committee rarely have the expertise to judge the quality of a candidate's research on their own. But this is EXACTLY the kind of power Annette and others are willing to hand over to members of the IRB, a committee typically with much LOWER standards for membership than a tenure & promotion committee. Regards, -- Christopher D. Green Department of Psychology York University Toronto, ON M3J 1P3 Canada 416-736-5115 ex. 66164 [EMAIL PROTECTED] http://www.yorku.ca/christo/ ================================ Jim Clark wrote: > Hi > > James M. Clark > Professor of Psychology > 204-786-9757 > 204-774-4134 Fax > [EMAIL PROTECTED] > > >>>> <[EMAIL PROTECTED]> 15-Oct-07 7:23 AM >>> >>>> > I think everyone who is critical of IRBs needs to sit on one. Once you see > the horrendous proposals that come through you will begin to see the logic of > the necessity to pay some attention to methodology. > > JC: > No Annette. This is an indication that there is something fundamentally > wrong with teaching, hiring, and evaluation processes in some academic > departments. It is not the job of IRBs to cover up for these inadequacies. > > Annette: > I completely agree with others about the need to examine the methodology, at > least for fundamentals. Having sat on our IRB for several years, primarily > because I was disgusted with some of their decisions, I quickly was able to > reach the conclusion that poor methodology, which clearly results in no gain > in knowledge is an abuse of human subjects. > > JC: > I fail to see how flawed studies constitute an abuse of subjects. In many > (most? all?) cases I would expect that subjects would not even be aware of > the flaws (see below). I think this is an example of how "harm" (i.e., > abuse) gets redefined in social science research to include things that it > was never intended to include in its original context. The reason this > happens, of course, is that there is no danger of any real harm, so "harms" > get invented in order to justify the ethics enterprise. > > Annette: > The risk value is NOT zero, it uses up the subject's time and energy that > could better be devoted to a more gainful use of time. Therefore there is an > investment that is going to be wasted, and will also prevent that person from > participating in a more useful study. This is not zero, it is a minus > situation for the risk factors. > > JC: > As above, I do not perceive these consequences as harms or risks in any > ethics-related sense of the terms. The threshold for "harm" has simply been > lowered from what should properly be the concern of IRBs (in disciplines > where real harms are possible). > > Annette: > Furthermore, at least at our university, we see participation in research as > part of learning about the research process. When students participate in > studies that even they can see are poorly designed, they come to distrust the > whole research process and its checks and balances. > > JC: > Do you have any evidence about what proportion of subjects participating in > flawed studies themselves identified the flaws? Any evidence that it > promotes distrust of the research process? One of the criticisms of IRBs is > that in fact evidence on most IRB-related issues has never been presented or > sought. And cannot students finding flaws in studies (including published > ones, of course) be useful for teaching rather than an impediment. Or does > pointing out the flaws of published studies in class undermine the research > process or instill in students a sense of the complexity of research and > competencies to identify weaknesses? > > Annette: > Most universities have a subject pool and if it is depleted by studies that > end up in the trash can then not only have you wasted the subjects' time but > you have denied good studies from being completed for lack of subjects. > > JC: > Isn't this something for departments to deal with, not IRBs? And wouldn't > departments be in a better position to evaluate whether its own studies are > methodologically sound and/or the teaching value of weak studies? These are > fundamentally not ethical issues that concern the protection of research > participants from harm. > > Annette: > Finally, the researcher with the poor study lost the benefit of some > constructive feedback, putting in tremendous effort and sometimes years of > work into something that ends up in the trashcan. > > JC: > Again, this is not the purpose of IRBs. Faculty mentors, administrators, and > perhaps even faculty associations might be concerned about such matters, but > by what rational are IRBs suppose to protect researchers from their own > idiocy, as opposed to protecting subjects from real harms that exceed those > met in daily life? > > Annette: > I have found, on every occasion when a weakness was pointed out in the > methods, that the researcher was grateful for the extra set of eyes. And it > was NEVER a matter of qualitative versus quantitative studies. > > JC: > That Annette has had such positive experiences probably says more about > Annette than about IRBs. And nothing in the criticism of IRBs's intruding > into research methodologies prevents researchers from soliciting > methodological comments from people qualified to provide such judgments. > > Annette: > It's all in one's attitude and experiences I guess, which is why I advocate > for anyone with negative feelings towards IRBs to sit on one. It will be > quite eye opening. > > JC: > We should not be basing our decisions about IRBs on our own experiences, but > on the collective experiences across the research community, including those > of people who have not sat on IRBs. Indeed, it may even be that non-IRB > researchers are a better source of objective information given the > psychological factors that could influence past and present IRB members > (group-think, cognitive dissonance, ...). > > Take care > Jim > > > --- > > > ====== ---
