"Bill Shipley" <[EMAIL PROTECTED]> writes: > Hello. I have come across a curious result that I cannot explain. > Hopefully, someone can explain this. I am doing a 1-way ANOVA with 6 > groups (example: summary(aov(y~A)) with A having 6 levels). I get an F > of 0.899 with 5 and 15 df (p=0.51). I then do the same analysis but > using data only corresponding to groups 5 and 6. This is, of course, > equivalent to a t-test. I now get an F of 142.3 with 1 and 3 degrees of > freedom and a null probability of 0.001. I know that multiple > comparisons changes the model-wise error rate, but even if I did all 15 > comparisons of the 6 groups, the Bonferroni correction to a 5% alpha is > 0.003, yet the Bonferroni correction gives conservative rejection > levels. > > How can such a result occur? Any clues would be helpful.
It's a question of assumptions. Notice first that you have some very small groups there. Comparing two groups with 3df means that there are five observations in all, presumably two in one group and three in the other (although it could be 4-1). The joint F test assumes that all the groups have a similar (theoretical) SD, whereas the two group comparison only assumes that those two groups are similar. Suppose one of the other groups had a huge SD; then a joint comparison would clearly lose power if the actual differences were between some of the groups with a smaller SD. On the other hand, the test on 3df is extremely dependent on distributional assumptions, and if data are non-normally distributed, there may be an increased probability of getting a very small variance (quantization can do that, e.g.) and thus a falsely significant result. I.e. I'd take a closer look at the SD's for the 6 groups and perhaps make a dotplot. -- O__ ---- Peter Dalgaard Blegdamsvej 3 c/ /'_ --- Dept. of Biostatistics 2200 Cph. N (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~~~~~~~~~ - ([EMAIL PROTECTED]) FAX: (+45) 35327907 ______________________________________________ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help
