On 12 Jan 2003 14:01:56 -0800, [EMAIL PROTECTED] (Karl L. Wuensch) wrote: > Monte Carlo work by Donald W. Zimmerman (Some properties of > preliminary tests of equality of variances in the two-sample location > problem, Journal of General Psychology, 1996, 123, 217-231) has > indicated that two stage testing (comparing the variances to determine > whether to conduct a pooled test or a separate variances test) is not > a good procedure, especially when the sample sizes differ greatly (3 > or 4 times as many subjects in one group than in the other, in which
... More or less, that is to say, "whenever it matters" > case the pooled test performs poorly even when the ratio of variances > is as small as 1.5). [break] I have done tests, some years ago, for my own information. My observations were not parameterized on the ratio of variances. I used data where the normals had been squared or exponentiated, to create skew-owing-to-the-need-for-transformation. In circumstances with skew, that I noticed where the pooled test performed "100% horrible", the non-pooled test was about "95% horrible". If I remember right, I saw the one-tailed 5% test perform badly while rejecting the null too seldom: (for instance) 0.8% versus 1.0% -- not a difference that makes *much* difference. [Oh, the t-test maintained its fabled *two-tailed* robustness, because both those examples rejected the *other* tail at about 9%.] > Zimmerman's advice is that the separate > variances t should be applied unconditionally whenever sample sizes > are unequal. Given the results of his Monte Carlo study, I think this > is good advice, [ break] Given my own Monte Carlo experiments, I concluded that a) using either test was far inferior to performing the proper transformation *first*, if there was proper, natural one; and b) using a rank-test was consequently going to be almost as superior, whenever there *is* a common transformation available; and c) I really have to re-consider my hypotheses, whenever neither (a) nor (b) is appropriate. Am I interested in variance differences? Am I really interested in "mean" or would I (does the client) rather focus on one extreme or another? - because for these data, the conclusions won't be consistent. Also, another consideration led me to conclude that the separate-variances was inferior for the scaled data that I analyze most often. That is: If you construct groups unequal proportions on a dichotomy, you will see unequal variances. With unequal Ns, you get different p-values when you carry out both t-tests - and the pooled tests are better approximations. Scales with 4 or 5 points seem to follow the same logic, and I have read an article or two that supported the same conclusion, about using the pooled t for those scales. > and I advise my students to adopt the practice of > using the separate variances test whenever they have unequal sample > sizes. But, I think, you have not showed *yourself* how wretched both tests can be. Doesn't the 1% and 9% surprise you? I'm convinced that whenever the two tests aren't consistent, you really ought to review your premises, about the whole problem. > I still believe that the pooled test may be appropriate (and > more powerful) when the sample sizes are > nearly equal and the variances not greatly heterogeneous, > but carefully defining "nearly equal sample sizes" and > "not greatly heterogeneous variances" is not something I care to tackle. -- Rich Ulrich, [EMAIL PROTECTED] http://www.pitt.edu/~wpilib/index.html . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
