Bruce Weaver wrote (in part):
...Negative priming is measured as a response time
> difference between 2 conditions in an experiment. The difference is
> typically between about 20 and 40 milliseconds...
> The researcher KNOWS that a lot of other things affect
> response time, and some of them a LOT more than his experimental
> conditions do. However, because one is interested in testing a theory of
> selective attention, this small difference between conditions is VERY
> important, provided it is statistically significant (and there is
> sufficient power); and measures of effect size are not all that relevant.
Where the measure of effect size is relevant here is in answer to the
question: Can we rule out all other plausible causes for what we observe? No
experimental design is perfect, and in real life one may be forced to work
with some that are very imperfect indeed. The experimenter may be able to
eliminate some major confounding variables by careful design; and while
there are always huge numbers of minor effects that *might* be confounded
with what one wants to observe, it is true that most of them are small in
size.
If one can conclude that the effect size is on the order of 20ms, one
can then ask oneself "is there anything else, not controlled for in the
experiment, that could cause an effect of that magnitude?" and with luck and
good management the answer would be "no". Whereas, if one just rejected the
null hypothesis, the corresponding question would be "is there anything
else, not controlled for in the experiment, that could cause an effect?"
and the answer, if given honestly, would be "yes".
In the case of negative priming, had the effect been of the order of 1ms
(and the sample size correspondingly vast), I would conjecture that many
other plausible causes (lengthened time between trials? more opportunity to
become curious about the experiment?) for the difference could be dreamed up
that would be difficult to eliminate.
-Robert Dawson
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
