Hello,
Dale makes much sense, IMHO. Performing a statistical test of a certain
hypothesis is a great place to start, but many leave it at that. Read almost
any clinical journal and you will see the same setup...say means for 2 groups
+/- s.d. (or sometimes s.e.) and then a p-value (N.S. if not significant and
report the p-value if it is...as if the p-value has some magical meaning if you
reject).
Very rarely do you see any discussion of the effect size. For a t-test,
reporting the difference +/- s.d. and a "95% CI" may give the reader a lot more
to go on. Say, for example, we measure weight in 2 groups of females. I use a
small sample and find a difference of 4.9 lbs. but "statistically" it is not
significantly different from 0. I report the means for the 2 groups and a
p-value and then state that there is not enough evidence to believe there are
differences. We are usually all suspicious of this...there is a pretty large
difference. Now, you repeat the experiment and use a large sample and find an
observed difference of 5.1 lbs and you report that it is highly significant
with p<0.0000000001.
The question is NOT whether it is statistically significantly different from
zero is it? Your larger sample will allow you to estimate the mean with more
precision. (I'm not sure that I agree that larger samples are more problematic
from the terms of missing data than smaller samples...that's a problem with
study design, not sample size.)
Warren May
Dale Berger wrote:
> When we focus on estimates of effect sizes and the stability of those
> estimates, we are delighted to have a huge sample. Don't focus on
> statistical significance.
>
> ===========================================================================
> This list is open to everyone. Occasionally, less thoughtful
> people send inappropriate messages. Please DO NOT COMPLAIN TO
> THE POSTMASTER about these messages because the postmaster has no
> way of controlling them, and excessive complaints will result in
> termination of the list.
>
> For information about this list, including information about the
> problem of inappropriate messages and information about how to
> unsubscribe, please see the web page at
> http://jse.stat.ncsu.edu/
> ===========================================================================
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
