Hi Rick,
I understand the authors' point and also agree that post-hoc power
analysis is basically not telling me anything more than the p-value and
initial statistic for the test I am interested in computing power for.
Beta is a simple function of alpha, p, and the statistic.
My intention is, as I mentioned in my response to Stephan Kolassa,
to transform my p-value and statistic into a form of effect size--sample
size necessary to attain significance at alpha=.05. This will communicate no
more information, it is just a mathematical re-representation of my data in
a way I believe my readers will find more informative and useful. In other
words, there is no more information *encoded*, but there is more information
*communicated,* just like for any effect size measure.
If you have any suggestions on a more reliable effect size for
MANOVA which is *also* commonly known in the social psychology community
(e.g., a correlation or Cohen's d analogue), I'm interested--but the
multivariate nature of the beast makes these more or less impossible to
translate.
The poster I was asking for is now printed, and we reported the
multivariate R-squared using the techniques in Cohen (1988), though I'm
expecting to spend a lot of time explaining what that means to people in a
multivariate context, rather than describing the results of the study.
Cordially,
Adam D. I. Kramer
On Sun, 1 Feb 2009, Rick Bilonick wrote:
On Wed, 2009-01-28 at 21:21 +0100, Stephan Kolassa wrote:
Hi Adam,
first: I really don't know much about MANOVA, so I sadly can't help you
without learning about it an Pillai's V... which I would be glad to do,
but I really don't have the time right now. Sorry!
Second: you seem to be doing a kind of "post-hoc power analysis", "my
result isn't significant, perhaps that's due to low power? Let's look at
the power of my experiment!" My impression is that "post-hoc power
analysis" and its interpretation is, shall we say, not entirely accepted
within the statistical community, see:
Hoenig, J. M., & Heisey, D. M. (2001, February). The abuse of power: The
pervasive fallacy of power calculations for data analysis. The American
Statistician, 55 (1), 1-6
And this:
http://staff.pubhealth.ku.dk/~bxc/SDC-courses/power.pdf
However, I am sure that lots of people can discuss this more competently
than me...
Best wishes
Stephan
The point of the article was that doing a so-called "retrospective"
power analysis leads to logical contradictions with respect to the
confidence intervals and p-values from the analysis of the data. In
other words, DON'T DO IT! All the information is contained in the
confidence intervals which are based on the observed data - an after the
fact "power analysis" cannot provide any insight - it's not data
analysis.
Rick B.
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
