Right on , Stuart. IMHO, "reliable" is as bad as "significant" when
referring to an effect for which p < .05. Whether an outcome is likely to
be replicated or not is solely a function of power, and most behavioral
research has little power. Given the power in typical psychological
research, the expected result when one attempts to replicate an outcome that
was "significant" earlier is failure to replicate -- that is, a Type II
error. You all should keep in mind that with continuous variables, no null
hypothesis is ever likely to be absolutely true, and with sufficiently large
sample size, an effect of miniscule size will be found to be statistically
"significant." Were we to try to replicate such an outcome with typical
sample sizes, we almost certainly would fail to replicate. Furthermore,
given typical sample sizes, and typical effect sizes, when we do find an
effect to be significant, it is largely a matter of good luck, of getting a
sample in which the apparent size of the effect is larger than it is in the
population. With a sample more representative of the population the effect
would not be statistically significant. We would all be better off if we
just dispensed with hypothesis testing and got on with estimating effect
sizes and putting confidence intervals about those estimates.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++ Karl L. Wuensch, Department of Psychology, East Carolina University,
Greenville NC 27858-4353 Voice: 252-328-4102 Fax: 252-328-6283
[EMAIL PROTECTED] http://core.ecu.edu/psyc/wuenschk/klw.htm
