Hi James M. Clark Professor of Psychology 204-786-9757 204-774-4134 Fax [email protected] Department of Psychology University of Winnipeg Winnipeg, Manitoba R3B 2E9 CANADA
>>> <[email protected]> 10-Jan-09 11:07 AM >>> On 9 Jan 2009 at 10:53, Mike Palij wrote: > It was for reasons > like this that Jack Cohen used to say that one-tailed tests should > not be done (how could one distinguish delusional self-confidence > from solid knowledge?) Absolutely. This isssue drives me nuts. One-tailed tests should be banned (almost), and certainly raise my suspicion whenever I see them used. But the issue isn't one of knowledge, and just because you make a prediction doesn't give you the right to use them. You must be able to swear (on the Bible, the Koran, or _The Design of Experiments_) that even if, contrary to all that you hold reasonable, the result falls in the "wrong" tail, you would have not the slightest interest in such an outcome, which you would declare to be totally meaningless. It's not easy to find such examples which fit this criterion allowing a one-tailed test. The only one I can recall is, say, that you're testing whether a therapy is better than a placebo. You predict that it is but, more important, you have absolutely no interest in an outcome which shows the therapy causes harm. Because harm and merely useless lead to the same conclusion: don't use the therapy. Even this isn't a good example, because a finding that a therapy causes deterioration could well have important implications. But a clinician might not care. JC: Would not a fairer characterization of one-tailed tests be that the unanticipated outcome would be a statistical artefact, rather than calling it "totally meaningless?" Would not that apply to a whole host of "possible" (but theoretically/empirically unexpected) outcomes, including: - better memory for abstract words than concrete words - people who are dissimilar to one another liking each other better than those who are similar - faster reading times for long, unfamiliar, irregular words than short, familiar, regular words - older children doing worse at an arithmetic task than younger children - children with ADHD doing better on an attention task or the stop-signal RT task ... In essence, does not every well-established effect constitute grounds for viewing a contradictory outcome as an artefact, and hence justify a one-tailed test? This does not, of course, deny the possibility that new knowledge (theoretical or empirical) could predict conditions that qualify the well-established finding (i.e., an interaction), perhaps including conditions where the effect would be reversed. Another important point relevant to this discussion is the importance of true replication in science, and the practice of meta-analysis to aggregate results across experiments. Everything (not much in the grand scheme of science??) does NOT hinge on a single result and a single statistical test. Take care Jim --- To make changes to your subscription contact: Bill Southerly ([email protected])
