This book is about #0 years out of date. As George Box said many years ago
(paraphrase) "This is akin to using a rowing boat to see if the sea is calm enough
for an ocean liner to set sail."
Jon Cryer
Box (1953) did say that, "To make the preliminary test on variances is rather like putting to sea in a rowing boat to find out whether conditions are sufficiently calm for an ocean liner to leave port!" (Box, G.E.P. (1953), "Non-normality and Tests on Variance," Biometrika, 40, 318 -335. ). Bot Box was also aware that the robustness of t and F to heterogeneity is heavily dependent on sample sizes.
In Box (1954) he wrote "It appears that if the groups are equal, moderate inequality of variance does not seriously affect the test. However, with unequal groups, much larger discrepancies appear." In that quote he meant "groups are equal" to refer to the sample sizes.
Books that advocate Satterthwaites solution, or Welch's, are certainly not out of date. It is still important to pay attention to heterogeneity. In fact, if the sample sizes are roughly equal, the corrected, and uncorrected, df will be nearly the same. If the sample sizes are quite unequal, the uncorrected test will be biased in a liberal or conservative direction, depending on the direction of the relationship between sample size and variance.
However, this does not directly speak to the question of conditioning a test on means on a test of variances. We know that heterogeneous variances with quite unequal n's can cause a problem. The question is "how do we resolve the problem?" When do we decide to adjust using a correction to the df--whether it be Satterthwaite, Welch, Greenhouse and Geisser, Huynh and Feldt, or whatever? The obvious answer is "when we have heterogeneity and unequal n's." But how do we know that if it isn't through looking at the data in some way. One way to look at the data is to use Levene's (1960) test (or a modification due to O'Brien,1981 or Brown and Forsythe, 1974) either as an index of disparity or a test of significance. But then you are conditioning on a test of variance. What alternative have you?
I would add that if your variances are heterogeneous, that is more than just a "pain in the neck" and a violation of underlying assumptions. If you provide therapy to one group, but not to another, heterogeneity is actually a finding as much as an assumption. If the variances are heterogeneous, and you have random assignment, your therapy must be doing something, regardless of what it does to the mean.
Dave Howell
New address:
David C. Howell
3007 Barton Point Circle
Austin, Tx 78733
http://www.uvm.edu/~dhowell/StatPages/StatHomePage.html
http://www.uvm.edu/~dhowell/gradstat/index.html
