I received e-mail which convinced me that my previous post in this
thread was not as coherent as I had hoped. Gene G. cited the part of
it that *was* quite all right, and gave a vivid example --
On Thu, 22 Jun 2000 16:31:24 GMT, Gene Gallagher
<[EMAIL PROTECTED]> wrote:
> Rich Ulrich wrote:
> > These are not quite equivalent options since the first one really
> > stinks -- If you are considering drawing conclusions about causation,
> > you need *random assignment* and the two Groups of performance are the
> > furthest thing from random.
> >
> > Let's see: the simple notion of regression-to-the-mean says that the
> > Best performers should fall back, the Worst performers should improve;
> > that's a weird main-effect, which should wreak havoc with interpreting
> > other effects.
> > Or: If the Pre is powerful enough to measure potential, then a
> > continued-growth model says that Best performers should improve more,
> > even given no treatments.
>
> This pattern was described in an obit about two-three years ago in the
> NY Times. A statistician's obit noted that he'd found a flaw in the
> Israeli air force's training program. Apparently, the Israeli air force
> was punishing the worst performers in a test because this usually
> produced a better performance in subsequent tests and was supposedly
> much more effective than positive reinforcement. They'd found that
> positive reinforcement of the best performers often resulted in a poorer
> performance on the next test. This now-deceased statistician pointed
> out the confounding effect of regression to the mean on this assessement
> of negative and positive reinforcement. The effectiveness of negative
> reinforcement (punishment) could be nothing more than a chance effect.
>
> I wish I had the citation for the study or the obit.
>
> Does anyone else in the group have a citation of this study?
[sorry, no citation.]
The original posting expressed an intent to compare Change scores for
subjects who were good on the Pre, versus bad. It is exactly *that*
comparison which is so difficult to do fairly. If it is a quick
followup, you have to take into account this regression to the mean.
IF it was a careful selection and a long followup then you might have
to take into account one factor of the other, regression-to-the-mean,
or the "regression away from the mean" which is a pragmatic feature
of some learning paradigms. AND it is not a feature of ordinary
statistics: You need to have multiple groups in order to recognize
it.
There were to be two groups. If these were randomized across
good/poor, there should be no problem with the main-effect tests. I
jumped beyond the stated problem to warn that there could be
interaction owing to bad scaling; that (of course) is always a
potential. I think I was moved to say something, though, because
"performance" is exactly where I have seen serious problems, in
various examples in the past.
Hope my previous post in this thread was not too confusing....
--
Rich Ulrich, [EMAIL PROTECTED]
http://www.pitt.edu/~wpilib/index.html
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
