- a not-so-quick comment on the document and on power-estimation. On 8 Jan 2003 18:14:53 -0800, [EMAIL PROTECTED] (Dennis Roberts) wrote:
> I worked on a small handout for my stat class about the margin of error > (like the +/- 3% used in polls) . While I fully acknowledge that I got the > idea from Moore and McCabe's book "The Practice of Statistics", I put it in > the context of examples used in my own classes. > > http://roberts.ed.psu.edu/users/droberts/introstat/ > > I think this notion ... is far too often OVERlooked ... and could be very > beneficial to the planning of some kinds of studies. > > The link above ... scroll down to marerr1.doc ... and RIGHT click to SAVE > TARGET AS (or you can double click and see what happens) ... it's a small > Word document. > > Comments appreciated. > I have found it very useful to anchor my notions of statistical power with the rule-of-thumb relationship: An exact replication of an experiment has only 50% power for achieving the p-value that was just achieved. That follows the common sense observation that if the exact experiment were done twice, there is no reason for the BETTER result to happen on the second try. (As statistical planners, we ignore the fact that the folks do try to do the 2nd experiment better. As planners, we also do have to limit the number of 'replications', or else control for multiple tests.) Exactly on this topic, I see a serious mis-statment in the document, under "Planning a study to yield a certain margin of error (M)". Where it says, "how large of a SAMPLE would I need, with 95% confidence, to produce an interval ...." - it ought to say, "... with 50% confidence, ..." since it is doing only the simplest extrapolation from the point-estimate of the SD the pilot study. That 95% was pulled out of thin air, or borrowed by accident (I guess) from the size of the CI. It is unaccounted for; it is a mistake. (But it is not surprising. This power stuff is tricky, and I think that I remember making that same mistake, back when I first started worrying about these, and before I found that "anchor.") Note that *if* the original SD was given as a population's exact value, then the original CI could be in terms of z; and reducing the size would be an exact formula, where cutting it in half requires exactly 4 times the relevant degrees of freedom. That is to say, it would be an EXACT formula, so the outcome for the CI would be guaranteed, rather than being "95%", or some other quantity. At the end of the document, it seems to recognize that, indeed, you can do something complicated (a power analysis) to accommodate the observed variability, to get to 95%. Here is how-to-do this one. Since the SD was given as an estimate based on 15 DF, the square of the standardized SD is "distributed as chisquared, 15 df" -- Now, that chisquared has a mean of 15, and its 95% upper limit is 25.0 When you work it out, that means the sample with 95% power needs 25 d.f. for each of the 15 d.f. in the original. Or, once the 15 have been extrapolated to 60 in order to reduce the CI, then 100 are needed, to have 95% power. I think that's all there is to it. In addition, the document overlooks the tentativeness of most real power statements; it misses the "iffy-ness." The statement of outcomes, I think, should be revised with attention to the difference between *observing* an effect-size of 0.1 mpg (What are these terms?), and *hypothesizing* something as being an *underlying* effect. That is: We know that the test with the CI of +/- 0.05 will REJECT if the observed effect is 0.05, not to mention 0.10. However, the usual "power statement" might attest that the power of the test to reject the null, given "some underlying 0.05 difference" is only 50%, since that is, indeed the 'critical size.' More particularly, the power statement says the power is 85% or 95% <whatever> to reject the null hypothesis, under these various conditions, if the hypothesized, underlying difference is 0.10. -- Rich Ulrich, [EMAIL PROTECTED] http://www.pitt.edu/~wpilib/index.html . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
