On 1 Dec 2003 05:54:53 -0800, [EMAIL PROTECTED] (Chuck Robertson) wrote: > I have a question about power analysis related to a t-test of some > weights in multiple regression equations. I'm afraid I'm a cognitive > psychologist, so if I give you to much info below about my experiment > please accept my apology. > [ snip; stuff that confused me ] > equations. As an example, at a 1 second encoding time the older adults > have a significant beta weight for speed as a predictor (b = .31) but > the young adults do not (b = .15), in the same equation young adults > have a significant beta weight for vocabulary (b = .24) but older adults > do not (b = .19). This casually looks like a difference when eyeballing > the equations, speed is the only significant predictor for older adults > and vocabulary is the only significant predictor for young adults at a 1
If I may speak as someone who has looked at a lot of regressions and has written a lot of power statements: If you don't have a really *strong* effect somewhere, then you aren't likely to detect *differences* from the effect. For a difference to show up on a .05 test, keep in mind that when the coefficient for one group is "p= 0.05", what it demonstrates is that the b is JUST BARELY determined as different from a FIXED VALUE of zero. If you want another test where a b is different from a another coefficient whose value was zero, that other test would not be 'as significant' -- comparing b to zero is going to be stronger than comparing b to a floating value that is only estimated as zero. So, for your tests: If one is positive and barely p=.05, then the other is going to have to be negative in order to have the test be significant. To me, <above> looks as if there are happenstance-differences showing up because a lot of tests are being performed; and (so far) it is rather unlikely that the data illustrate any actual differences between the ages. If the same group were favored in both tests that gave p < .05, then there *might* be a bit more reason to think that one group was superior -- in my experience, 'main effects' are more likely and more believable than 'interactions'; having opposing effects speaks to an interaction among variables. > second encoding time for the memory material. Interestingly, when I look > at the t-test for difference between the B weights, I fail to reject the > null of no significant difference between the weights. The t-test for > difference between the speed predictors is t = -.51 and t = .14 for the > vocabulary predictors. Of course I do not want to accept the null, I am > really curious as to how much power I had to detect a significant > difference with my sample size of 80 younger and 80 older adults in each > equation. I hesitate to give any formula because you thoroughly do not understand what you have in hand. Let's say that you power comes out to be (say) 15% and 8% for those two. Now what? > > I need help computing the effect size given my SPSS output from the > regressions. I assume that I need to compute f as an effect size since > this isn't teh typical t-test. Can anyone tell me how to do that given > the weights and the standard errors for the weights from the output? I > must be doing something wrong as my f's are way to large. > Well, I should back up and tell you this: So far as I could tell, you did not use the appropriate test. I'm still not sure what you are working at, but I'm pretty sure that you should have -- a) tested a model with all 160 cases; and b) added a variable to account for one group's difference in intercept (look at its test) ; and c) added another variable (and test) to account for slope. Your hypothesis is probably something like one of them, (b) or (c), where the other one is a nuisance parameter. But I could be wandering off base, so I will not say any more for now. -- Rich Ulrich, [EMAIL PROTECTED] http://www.pitt.edu/~wpilib/index.html "Taxes are the price we pay for civilization." . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
