John W. Kulig wrote:
> Sorry this is going to the general tips group rather than tips-methods, but I
> haven't seen much on the methods list and don't know if that list is still active.
> You have a simple two-group (independent) situation, and do an significance
> test contrasting the two means (t or F). You have several other studies doing
> the same thing. To combine them into one overall test for significance,
> Rosenthal and Rosnow (1991) recommend getting Z scores from the p levels
> reported in each study. But, you have to use the _one-tail_ probabilities for
> the meta-analysis. This is easy if t was reported. If t was reported as
> two-tailed, just half the p reported. So if they report t = such-and-such and
> two-tailed p = .08, .04 is the one-tailed value. But what about F? The F
> distribution is naturally one-tailed (well, there is a little area to the left
> which might represent the "true good to be true" reject, but that is of little
> significance). It would be improper to "half" the reported p value from ANOVA,
> but, isn't it also the case that if we simply changed the reported F to a t
> (since F = t^2), and _then_ half the p to represent only one-tail, we are
> doing the same thing? If that's the case, then is it true (for practical
> purposes only) that we can simply half the reported p from F and consider it
> as if it were a one-tailed directional test, in those cases where only two
> groups were used? After all, under Null where mean 1 = mean 2, half of the
> cases in the rejection area of the F distribution will be when mean 1 > mean
> 2, and the other half when mean 1 < mean 2. Or am I missing something?
>
> --
> * John W. Kulig, Department of Psychology ************************
> * Plymouth State College Plymouth NH 03264 *
> * [EMAIL PROTECTED] http://oz.plymouth.edu/~kulig *
> *******************************************************************
> * "Eat bread and salt and speak the truth" Russian proverb *
> *******************************************************************
John,
My standard reference for this question is "On telling tails when combining results
of independent studies" by Robert Rosenthal, from Psychological Bulletin, 1980, vol 88,
pages 496-497. I am sure there are other good discussions of the issue, but this is
brief and to the point.
This article points out that an F test is one-tailed, but this does not mean that the
hypothesis being tested is one-tailed. The confusion comes from the two meanings of
the term "one-tailed." First, it describes the F distribution. Secondly, it describes
the research hypothesis. With these two meanings in mind, you can see that you can
test a two-tailed hypothesis with a one-tailed F test.
Rosenthal also points out that the same logic applies to the chi-square test. Like
the F test, it also has a one-tailed distribution.
Rosenthal then gives an example that I have found useful in making the point about
tails when teaching. Students are pretty clear that the cutoff for a one-tailed
(directional) z test is 1.65 and that the cutoff for a two-tailed (non-directional) z
test is 1.96. With this in mind, remind the students that a chi-square with one degree
of freedom is equal to z squared. Here comes the fun part. If we square a z of 1.96,
we get a chi-square of 3.84. Look in any chi-square table, and you will see that a
chi-square of 3.84 is labeled as significant at the .05 level. In other words, this
chi-square test that is distributed as one-tailed is testing a two-tailed hypothesis.
I hope this has been helpful.
Dave Kerby
Dept of Psych
Northeast Louisiana Univerisity
Monroe, LA
