In addition to Dennis' contribution, I would point out a couple of things about the robustness of the ANOVA (or lack thereof):
First, a fixed-effects ANOVA can have a quite inflated Type I error rate if the groups are unequal in size. In that case, you wouldn't want to use Tukey; you'd want the Games-Howell procedure, which is designed for use with unequal n's. I had a data set recently in which the n's were unequal and the F indicated a significant difference in means, but the Games-Howell procedure (more "trustworthy" in this case than the ANOVA) showed no significant differences. So the F results probably was a Type I error. Second, even if you have equal and large n's, severely unequal variances can lead to increased probability of a Type I error. So you might find significance with the F test but nothing significant with the post-hoc test, as I found in the unequal n's case above. If your n's and variances are unequal, and if your smaller n is paired with the smaller variance, the Type I error rate tends to be conservative -- that is, not detect a difference that actually is there. So you'd get a non-significant F and you'd want to proceed with the Games-Howell at that point. To me, the ANOVA result often is so general (either telling you "there's no difference in there -- but that could be wrong" or "there's a difference in there -- but that could be wrong") that the post-hoc comparisons are the ones that really will answer the questions you're interested in. Also, if you've got a one-way ANOVA with equal n's, you might consider using the Ryan (REGWQ) procedure, which has a power advantage over Tukey. (Toothaker has a book available from Sage on MCPs.) Cheers. Lise -----Original Message----- From: Ivan Balducci <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Date: Thu, 19 Dec 2002 09:34:23 -0200 Subject: A question about ANOVA and Post Hoc test Hi members, Please, may someone explain to me this doubt: why is not right to perform a Tukey HSD test after an ANOVA came out non-significant ? Why Tukey HSD test should be done only following an ANOVA that was significant ? Which is the reason ? TIA, Ivan Balducci . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
