Suppose that you have three groups. If the omnibus null is true, the probability of erroneously rejecting the null with the overall Anova is equal to alpha, which I'll assume you set at .05. IF you reject the null, you have already made one type I error, so the chances of making more do not matter to the familywise error rate. Your Type I error rate is .05.
Now suppose that the null is false-- mu(1) = mu(2) /= mu(3). Then it is not possible to make a Type I error in the overall F, because the omnibus null is false. There is one chance of making a Type I error in testing individual means, because you could erroneously declare mu(1) /= mu(2). But since the other nulls are false, you can't make an error there. So again, your familywise probability of a Type I error is .05.
Now assume 4 means. Here you have a problem. It is possible that mu(1) = mu(2) /= mu(3) = mu(4). You can't make a Type I error on the omnibus test, because that null is false. But you will be allowed to test mu(1) = mu(2), and to test mu(3) = mu(4), and each of those is true. So you have 2 opportunities to make a Type I error, giving you a familywise rate of 2*.05 = .10.
So with 2 or 3 means, the max. familywise error rate is .05. With 4 or 5 means it is .10, with 6 or 7 means it is .15, etc.
But keep in mind that, at least in psychology, the vast majority of experiments have no more than 5 means, and many have only 3. In that case, the effective max error rate for the LSD is .10 or .05, depending on the number of means. Other the other hand, if you have many means, the situation truly gets out of hand.
Dave Howell
At 10:37 AM 2/8/2002 -0800, you wrote:
Hello, I have two questions regarding multiple comparison tests for a one-way ANOVA (fixed effects model).
1) Consider the "Protected LSD test," where we first use the F statistic to test the hypothesis of equality of factor level means. Here we have a type I error rate of alpha. If the global F test is significant, we then perform a series of t-tests (pairwise comparisons of factor level means), each at a type I error rate of alpha. This may seem like a stupid question, but how does this test preserve a type I error for the entire experiment? I understand that with a Bonferroni-type procedure, we can test each pairwise comparison at a certain rate, so that the overall type I error rate of the experiment will be at most a certain level. But with the Protected LSD test, I don't quite see how the comparisons are being protected. Could someone please explain to me the logic behind the LSD test?
2) Secondly, are contrasts used primarily as planned comparisons? If so, why?
I would very much appreciate it if someone could take the time to explain this to me. Many thanks.
Go Get It!
Send FREE Valentine eCards with Lycos Greetings
http://greetings.lycos.com
=================================================================
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at
http://jse.stat.ncsu.edu/
=================================================================
David C. Howell
Dept of Psychology
University of Vermont
Burlington, VT 05405
http://www.uvm.edu/~dhowell/StatPages/StatHomePage.html
http://www.uvm.edu/~dhowell/gradstat/index.html
