My response is about regression to the mean generally, which got done
over a little over a week ago.
It occurred to me recently that you could reduce the
regression-to-the-mean effect by using the subjects' least-squares
means to divide them (the subjects) up into quantiles for separate
analysis (or use the quantiles as a covariate in the model). That
would reduce the within-subject error by a factor of 1/root2, if
there is a single pre and post test, so the RTM effect would be
reduced accordingly.
See http://trochim.human.cornell.edu/kb/regrmean.htm for an estimate
of the magnitude of RTM. It is an amazingly simple 1-r times the
distance from a pre-test score to the pre-test mean, where r is the
intraclass correlation coefficient, obtained from a reliability study
(or, next best thing, obtained by treating the experiment as a
reliability study). In other words, the expected drift towards the
mean is 1-r times the difference between the pre-test score and the
mean score. If r=0, expect to regress fully to the mean. If r=1,
expect no regression to the mean. Trochim explains how to take into
account real shifts in the mean in the post-test. I hope all this is
not old hat for this list.
I am grateful to Greg Atkinson at Liverpool John Moores University
for pointing me to the above website, in case he ever sees this
message (he's not on this list). By searching back I see that Gene
Gallagher <[EMAIL PROTECTED]> also referred to this site in Re:
AW: MA MCAS statistical fallacy. I haven't read the other responses
on the MCAS statistical fallacy, so I hope I am not repeating anyone
else's ideas here.
The expression for r is (between^2 - within^2)/between^2, where
between and within refer to the usual between-subject SD, and within
is the within-subject error (standard deviation). With a bit of
algebra you can show that the RTM effect using the least-squares mean
of two tests will be (1-r)/2, that is, half the usual value. If you
have four tests altogether (e.g., 2 pre, 1 mid, 1 post), then other
things being equal, the RTM effect will be quarter what it is if you
use just the pre-test.
I've never done it, but I presume you just subtract off the estimated
effect of RTM when you want to take account of it. Putting
confidence limits on the result will be difficult, I imagine, unless
you use bootstrapping.
By the way, for those who responded to my query about non-normality
of residuals, thanks heaps again. I am in the middle of some
simulations. It's taking me a while, because I found that Proc Mixed
fell over when the variance was zero in a subgroup. Proc Ttest
didn't, but it didn't give confidence limits based on unequal
variances, so I have had to generate those myself from the output
using version 8 SAS. I had been using SAS version 6.12 up until now,
but that didn't generate the required output, nor did it lend itself
to simulation with Proc Ttest. Preliminary findings: that magic
sample size of 30...
Will
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
