Stephen Black wrote:
> 2) Multiple comparisons
>
> We should be warning students of the error of uncorrected
> multiple comparisons, and instructing them how to mend their
> ways. I recommend the Bonferroni, an intuitively easy-to-grasp,
> easy to do, and universally applicable method. Its chief
> drawback, alleged excessive conservatism, is actually an
> advantage. Because the correction is more severe the more
> comparisons are made, it encourages limiting comparisons to only
> those which are truly meaningful. Moreover, its use would cut
> down on findings which aren't.
I agree that multiple comparisons by themselves are not the best way to
do science. Bonferroni is a strict cure, however. To me it's a question of
balancing Type I and Type II errors. If you get too strict with Bonferroni
you may throw out something good. Keppel's _Design and Analysis_ (1991) text
recommends a third category of decision - between reject the null, and fail
to reject the null. Use this category when you have a result significant at
the .05 level but not at the Bonferroni-corrected level. I believe he calls
it a "suspend judgment" category. This keeps a marginal result alive until
you (or another researcher) can replicate, and turn the serendipitious
finding of one study into the planned comparison of another ....
--
---------------------------------------------------------------
John W. Kulig [EMAIL PROTECTED]
Department of Psychology http://oz.plymouth.edu/~kulig
Plymouth State College tel: (603) 535-2468
Plymouth NH USA 03264 fax: (603) 535-2412
---------------------------------------------------------------
"What a man often sees he does not wonder at, although he knows
not why it happens; if something occurs which he has not seen before,
he thinks it is a marvel" - Cicero.
