It's not always so easy. I just did a study that used reading comprehension as the DV which was a 20-item quiz over a set of readings. The material was intense and unfamiliar and on avereage students scored 13 on the quiz. But a few scored only 4 or 5 points. Were they not careful readers or were they less capable comprehenders? I know one person is a student I have had in 2 classes and she is as honest and hard working as the day is long but not a very bright student. Another was someone I dont know who rolled his eyes at the task, asked if he really was expected to do this, and was there really going to be a follow-up and finished the task in less than half the usual time.
I can make assumptions about the two--the first did very poorly because that is the best she can do; the second did very poorly because it was clear he was not motivated. But how do I make that decision ethically other than by my gut feeling? So, this is not an easy question and I may drop both as outliers. Annette Annette Kujawski Taylor, Ph.D. Professor of Psychology University of San Diego 5998 Alcala Park San Diego, CA 92110 619-260-4006 [email protected] ---- Original message ---- >Date: Thu, 07 May 2009 12:23:46 -0500 >From: "Jim Clark" <[email protected]> >Subject: Re: [tips] educating participants in research >To: "Teaching in the Psychological Sciences (TIPS)" <[email protected]> > >Hi > >It certainly would be nice for all students to take research participation >(and class participation and tests and life and ...) equally serious, but that >is unlikely to ever be the case. I doubt, however, that slack participants >have much effect. Only a few obvious ways that they could affect the results >(off the top of my head): > >1. Putting down same response for all items. Would affect mean of scale(s), >depending on response emitted and average of responses. No effect on >differences between scales administered to all participants or on different >experimental conditions for within-s factors. Perhaps an effect for between-s >factors, depending on proportion of such respondents, their allocation to >condition, and their chosen response. Primarily "noise" added to between-s >SSs? No effect on reliability or validity of measures? > >2. Responding randomly. Would primarily add additional noise to within-group >SSs (error) for between-s factor. Negative effect on reliability and validity >of measures? > >3. Identifying purpose of study and responding to promote or negate "expected" >results. Probably more effort than simply participating honestly in study. > >There are ways to identify participants who could be excluded (as one poster >suggested) or to minimize their impact. > >1. For reaction times, exclude participants with too many unreasonably fast or >slow trials. I think the IAT does something like this. > >2. Positively and negatively worded questions? > >3. MMPI and other tests have ways to catch random responding that might be >used (e.g., too many conflicting responses to "identical" questions). > >4. Easy to screen for people who do not generate variable responses. > >Perhaps also worth noting that if this were a serious problem, then one would >NOT find predicted relationships or produce consistent results across studies. > I suspect most students take the task as seriously as it merits (it is not >life and death) given they are going to spend time at it and produce >worthwhile data. > >Take care >Jim > > > >James M. Clark >Professor of Psychology >204-786-9757 >204-774-4134 Fax >[email protected] > >Department of Psychology >University of Winnipeg >Winnipeg, Manitoba >R3B 2E9 >CANADA > > >>>> "Blaine Peden" <[email protected]> 06-May-09 1:47 PM >>> >Our students and faculty conduct research with participants from introductory >psychology and other courses. Some participants seem to do the studies in >great haste and with little sincerity and thereby raise concerns about the >quality of their data. Have you developed strategies or instructional >materials that explain the process and purpose of psychological research to >future participants and also promotes their involvement and integrity? I >welcome any comments, suggestions, or resources. > >thanks so much, blaine >--- >To make changes to your subscription contact: > >Bill Southerly ([email protected]) > > >--- >To make changes to your subscription contact: > >Bill Southerly ([email protected]) --- To make changes to your subscription contact: Bill Southerly ([email protected])
