Stephen,
I agree completely with your 3rd recommendation. If we must use NHST, then
at least we should accept that we are dealing with a conditional
distribution based on the null hypothesis being true. Our opinions about
the direction in which the null hypothesis could be false is irrelevant,
except under the rare situations in which it could be false in only one
direction.
However, recognition of this seems inconsistent with your first
recommendation. Is it possible that there are limited numbers of ways in
which a possible set of null hypotheses could be wrong? Should we ignore
those differences that do not fit our theoretical models?
- Mike
--
Stephen Black wrote:
> I've never taught statistics, another thing I'm thankful for
> (see my previous comment on pregnancy).
>
> But there are three specific, widely neglected topics that I wish
> people who do teach statistics would give some attention to.
> Especially when their treatment supports my own views on these
> topics, as follows:
>
> 1) Planned vs post-hoc "shotgun" approaches to pairwise
> comparisons.
>
> We should be encouraging students to consider beforehand
> the set of comparisons which are necessary and sufficient to
> extract all needed information from an experiment, and then to
> test only those comparisons (the planned as opposed to the
> post-hoc "shotgun" approach).
>
> 2) Multiple comparisons
>
> We should be warning students of the error of uncorrected
> multiple comparisons, and instructing them how to mend their
> ways. I recommend the Bonferroni, an intuitively easy-to-grasp,
> easy to do, and universally applicable method. Its chief
> drawback, alleged excessive conservatism, is actually an
> advantage. Because the correction is more severe the more
> comparisons are made, it encourages limiting comparisons to only
> those which are truly meaningful. Moreover, its use would cut
> down on findings which aren't.
>
> 3) One-tailed vs two-tailed tests
>
> Having a directional hypothesis is insufficient justification for
> the use of a one-tailed test. We should make students swear never
> to use one-tailed tests except in very, very, special
> circumstances.
>
> -Stephen
>
> ------------------------------------------------------------------------
> Stephen Black, Ph.D. tel: (819) 822-9600 ext 2470
> Department of Psychology fax: (819) 822-9661
> Bishop's University e-mail: [EMAIL PROTECTED]
> Lennoxville, QC
> J1M 1Z7
> Canada Department web page at http://www.ubishops.ca/ccc/div/soc/psy
> Check out TIPS listserv for teachers of psychology at:
> http://www.frostburg.edu/dept/psyc/southerly/tips/
> ------------------------------------------------------------------------
*****************************************************************
* Mike Scoles * [EMAIL PROTECTED] *
* Department of Psychology * voice: (501) 450-5418 *
* University of Central Arkansas * fax: (501) 450-5424 *
* Conway, AR 72035-0001 * *
********* http://www.coe.uca.edu/psych/scoles/index.html ********
