Dear Tipsters, Don McBurney was good enough to jump in and, because I was the one who invoked his names, I would like to add a comment.
I like his gold standard of two requirements: manipulate the IV and control those peskt confounding (confounded, extraneous) variables. This leaves open the question of how the requirements are met. If you have a within-subjects design, say in a memory experiment for recall of concrete and abstract words, you can meet both: Randomize the concrete and abstract words on the list and equate the words on variables other than concreteness (e.g., frequency, length, meanigfulness). Score differences on recall between concrete and abstract can then be attributed to this independent variable. The key point is to be able to have some confidence that you can say that the IV caused changes in the DV. This is not to say that nonexperimental methods are bad science. Sometimes you have no choice and many interesting relationships have been discovered with them. We only have to be careful about the kinds of conclusions we draw. Don has shown in his book that in some circumstances we can probably say that a treatment had an effect even though it was a quasi experiment. Sincerely, Stuart Date sent: Wed, 26 Feb 2003 13:29:48 -0500 From: "Donald H. McBurney" <[EMAIL PROTECTED]> Subject: Re: We've got it all wrong To: "Teaching in the Psychological Sciences" <[EMAIL PROTECTED]> Organization: University of Pittsburgh Send reply to: "Teaching in the Psychological Sciences" <[EMAIL PROTECTED]> > I hesitate to chime in here, because I agree with most of what has > been > said. Also, as my name has been invoked as an authority of sorts, I > am afraid of muddying the water. But if I can step back and speak > without trying to be "authoritative," it seems to me that all of the > distinctions we make-- between experiments, quasi-experiments, > nonexperimental work, etc.-- is part of an attempt to bring some order > to a wide range of activities. The categories are somewhat fuzzy, in > my opinion. Certainly randomization and manipulation are the gold > standard. But it is hard to argue that (e.g.) psychophysical research > using one or two subjects (in a within design) isn't true > experimentation. > One not-so-minor point: The purpose of randomization is not to > produce > groups that are approximately equal--matching can do a much better > job. 1) Randomization is the only method that guarantees that there > is no confounding except by chance, and 2) the probability that > differences between groups occured by chance can be assessed by > inferential statistics (which assume the random model). > best > don > Donald McBurney > University of Pittsburgh > > jim clark wrote: > > > Hi > > > > On Wed, 26 Feb 2003, Hatcher, Joe wrote: > > > > > I would argue that random assignment is only a means to > > > an end, the end being having at least two groups that are > > > assumed to be roughly equal on all variables. Seen that way, a > > > within-subjects design is simply another means of achieving the > > > same end. I understand the limitations, but would argue that with > > > appropriate counter-balancing these can be overcome. This may be > > > an argument of convenience for me or a result of > > > dissonance-reduction on my part; we have a very small subject pool > > > and strongly encourage within-subject designs where possible > > > > I would agree with Joe here. Moreover, within-subject designs > > are certainly the dominant design in many areas of psychology > > that have a (perhaps undeserved according to Mike?) reputation as > > being experimental (e.g., cognitive, perception, ...). > > Within-subject designs also generally increase the power of the > > design/analysis, as the error term is usually smaller when > > individual differences in subjects are reduced. > > > > > > From: Mike Scoles > > > > I'm going to stick with Sir Fisher and reserve the term > > > "experiment" for > situations where there is random > > > assignment to conditions. I do not know > of > any > > > within-subjects designs that would not be better as mixed > > > designs. > Within-subject designs are too easily compromised > > > by history, maturation, > instrumentation, attrition, and > > > (sometimes) test sensitization and > regression issues. > > > Let's see, the only one of the "Big 7" that I left out > was > > > subject selection--the major problem with quasi-experiments. > > > Of > course, > good quasi-experiments can provide information as > > > useful as a marginal > experiment. > > > > If you think of it as random assignment of conditions to subjects > > (rather than the reverse) then within-subject designs can be > > accommodated. That is, it is possible make the conditions > > orthogonal to many (all?) potentially confounded variables in > > within-subject designs. > > > > I would also extend Mike's last statement even further. > > Non-experimental designs can also provide information as useful as > > (or more useful than) marginal or poorly designed experiments. Many > > students appear to believe (wrongly, I think) that an experiment > > necessarily means causal inferences are warranted, whereas in fact > > only well-designed experiments permit strong causal inferences. > > > > Best wishes > > Jim > > > > ==================================================================== > > ======== James M. Clark (204) 786-9757 > > Department of Psychology (204) 774-4134 Fax > > University of Winnipeg 4L05D Winnipeg, Manitoba > > R3B 2E9 [EMAIL PROTECTED] CANADA > > http://www.uwinnipeg.ca/~clark > > ==================================================================== > > ======== > > > > --- > > You are currently subscribed to tips as: [EMAIL PROTECTED] > > To unsubscribe send a blank email to > > [EMAIL PROTECTED] > > > > --- > You are currently subscribed to tips as: [EMAIL PROTECTED] > To unsubscribe send a blank email to > [EMAIL PROTECTED] ___________________________________________________ Stuart J. McKelvie, Ph.D., Phone: (819)822-9600 Department of Psychology, Extension 2402 Bishop's University, Fax: (819)822-9661 3 Route 108 East, Lennoxville, e-mail: [EMAIL PROTECTED] Quebec J1M 1Z7, Canada. Bishop's University Psychology Department Web Page: http://www.ubishops.ca/ccc/div/soc/psy ___________________________________________________ --- You are currently subscribed to tips as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED]
