Dennis Roberts writes:
>you set your sights on a power of .7 ... (beta therefore being .3) ...
>select a two tailed alpha of .05 ... because the situation is such that
>this program could actually make you gain weight .... though you hope that
>it will help you lose weight
>
>now, let's assume that you want to detect an effect of 3 pounds ... either
>gain or loss ... and you therefore go about estimating the n needed to
>achieve this goal of being able to reject the null with a p of .7 ... if in
>fact the null is not true ... and the gap between the null and the center
>of the treatment effect distribution being 3 ...
>
>now, what if you execute your study rigorously with the n you estimated you
>would need ... and then reject the null with a p = .02 (for illustration
>purposes only) ... at the moment, don't worry if it is a gain or loss ...
>just that you reject the null
>
>here is my question (you were wondering when i would get to it, right?)
>
>WHAT CAN WE SAY, BASED ON THIS REJECTION OF THE NULL, about the treatment
>effect being 3 lbs OR more ... ?
>
>what confidence do we have that the treatment effect is AT LEAST 3 lbs?
Since the standard deviation that you used to estimate power is not the same
standard deviation that you are now using to get your p-value, you can't say
much of anything.
A simple confidence interval would answer your question well. Let's suppose
the 95% CI is (1.2,2.6). That tells you something quite obvious about
whether the actual weight change is more than 3 pounds. So would an interval
like (4.2,6.8). An interval like (2.2,4.6) would be a bit more uncertain.
Now if the standard deviation in the power calculation matched up well with
the standard deviation you get with the real data, you might be able to say
something with a bit more authority, but I'd have a bit of trouble figuring
it out. With real data, why wouldn't you just compute a confidence interval
to answer your question instead of worrying about exactly what your a priori
power calculation tells you about practical significance?
Steve Simon, [EMAIL PROTECTED], Standard Disclaimer.
STATS - Steve's Attempt to Teach Statistics: http://www.cmh.edu/stats
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
