On Mon, 11 Aug 2003 11:13:12 -0400, Rich Ulrich <[EMAIL PROTECTED]>
wrote:
>On Mon, 11 Aug 2003 09:35:19 -0400, Ken Butler
><[EMAIL PROTECTED]> wrote:
>
>> On Mon, 11 Aug 2003 12:07:35 +0200, Piotr Wi¶niewski <[EMAIL PROTECTED]>
>> wrote:
>>
>> >t-Student test
>> >log-transformed data
>> >------------------------
>> > n mean SD
>> >X1 30 0.26 0.46
>> >X2 6 -0.11 0.17
>> >
>> >p=0.17
>>
>> Are you using the "pooled" t-test that assumes equal population
>> variances for the two groups?
>>
>> If you are, this situation, where the smaller group also has the
>> smaller SD, can give very misleading P-values.
>
>That last comment is misleading, too. It implies
>that the *other* t-test is not (approximately)
>just as flawed. Which it is.
If by "other t-test" you mean where the smaller sample has the larger
of two notably unequal SDs, then I agree (the P-value will then tend
to be too small).
But if you mean the Welch t-test (where equal variances are not
assumed, and the P-value is an approximation based on a choice of df
to make a t-distribution fit well), then my agreement is rather more
qualified. There are two (or more) issues:
- robustness: what happens to P-values when the populations are not
normal? The answer here is probably "bad things" whether you use the
pooled-variance or Welch tests.
- type I error probabilities when the populations are normal. My
understanding is that it's only the pooled test that suffers here,
something sort-of confirmed by a few texts and confirmed rather more
dramatically by a little simulation.
I generated random samples of sizes 30 and 6 from normal distributions
with SDs 0.46 and 0.17 (to echo the figures quoted above) and means
both 0. I first did 1000 runs using the pooled test, and found that:
I rejected at alpha=0.10 0.008 of the time
0.05 0.003 of the time
0.01 0.000 of the time
The P-values are way too big, and we are rejecting way too
infrequently. This is what I expected.
Compare this with the results of 1000 runs of the Welch test:
I rejected at alpha=0.10 0.094 of the time
0.05 0.045 of the time
0.01 0.011 of the time
There's no suggestion here of any systematic effect on P-values.
>I notice that the SDs are rather different for the
>log-transformed numbers, above, which does not
>lead one to regard the log fixing the matter.
Indeed. Though the original poster said (if I understand correctly)
that the log-transform made the data in the separate groups look more
normal-shaped, which is one step forward. (How you assess normality
with a sample size of 6 is another question.)
Cheers,
Ken.
--
Ken Butler, Lecturer (Statistics)
University of Toronto at Scarborough
butler (at) utsc.utoronto.ca
http://www.utsc.utoronto.ca/~butler
.
.
=================================================================
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at:
. http://jse.stat.ncsu.edu/ .
=================================================================
