On 27 Sep 2001, Paul R. Swank wrote:
> Some years ago I did a simulation on the pretest-posttest control group
> design lokking at three methods of analysis, ANCOVA, repeated measures
> ANOVA, and treatment by block factorial ANOVA (blocking on the pretest using
> a median split). I found that that with typical sample sizes, the repeated
> measures ANOVA was a bit more powerful than the ANCOVA procedure when the
> correlation between pretest and posttest was fairly high (say .90). As noted
> below, this is because the ANCOVA and ANOVA methods are approaching the same
> solution but ANCOVA loses a degree of freedom estimating the regression
> parameter when the ANOVA doesn't. Of course this effect diminshes as the
> sample size gets larger because the loss of one df is diminished. On the
> other hand, the treatment by block design tends to have a bit more power
> when the correlation between pretest and posttest is low (< .30). I tried to
> publish the results at the time but aimed a bit too high and received such a
> scathing review (what kind of idiot would do this kind of study?) that I
> shoved it a drawer and it has never seen the light of day since. I did the
> syudy because it seemed at the time that everyone was using this design but
> were unsure of the analysis and I thought a demonstration would be helpful.
> SO, to make a long story even longer, the ANCOVA seems to be most powerful
> in those circumstances one is likely to run into but does have somewhat
> rigid assumptions about homogeneity of regression slopes. Of course the
> repeated measures ANOVA indirectly makes the same assumption but at such
> high correlations, this is really a homogenity of variance issue as well.
> The second thought is for you reviewers out there trying to soothe your own
> egos by dumping on someone else's. Remember, the researcher you squelch
> today might be turned off to research and fail to solve a meaty problem
> tomorrow.
>
> Paul R. Swank, Ph.D.
> Professor
> Developmental Pediatrics
> UT Houston Health Science Center
>
Paul's post reminded me of something I read in Keppel's Design and
Analysis. Here's an excerpt from my notes on ANCOVA:
Keppel (1982, p. 512) says:
If the choice is between blocking and the analysis of covariance, Feldt
(1958) has shown that blocking is more precise when the correlation
between the covariate and the dependent variable is less than .4, while
the analysis of covariance is more precise with correlations greater than
.6. Since we rarely obtain correlations of this latter magnitude in the
behavioral sciences, we will not find a unique advantage in the analysis
of covariance in most research applications.
Keppel (1982, p. 513) also prefers the Treatments X Blocks design
to ANCOVA on the grounds that the underlying assumptions are less
stringent:
Both within-subjects designs and analyses of covariance require a number
of specialized statistical assumptions. With the former, homogeneity of
between treatment differences and the absence of differential carryover
effects are assumptions that are critical for an unambiguous
interpretation of the results of an experiment. With the latter, the most
stringent is the assumption of homogeneous within-group regression
coefficients. Both the analysis of covariance and the analysis of
within-subjects designs are sensitive only to the linear relationship
between X and Y, in the first case, and between pairs of treatment
conditions in the second case. In contrast, the Treatments X Blocks
design is sensitive to any type of relationship between treatments and
blocks--not just linear. As Winer puts it, the Treatments X Blocks design
"is a function-free regression scheme" (1971, p. 754). This is a major
advantage of the Treatments X Blocks design. In short, the Treatments X
Blocks design does not have restrictive assumptions and, for this reason,
is to be preferred for its relative freedom from statistical assumptions
underlying the data analysis.
--
Bruce Weaver
E-mail: [EMAIL PROTECTED]
Homepage: http://www.angelfire.com/wv/bwhomedir/
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
