On Thu, 03 Aug 2000 10:34:24 GMT, [EMAIL PROTECTED] wrote:
> Hello from Germany,
> as a part of my dissertation in medicine, I have
> to summarize some results of clinical trials.
> My question: By summarizing the results
> (percentage differences of certain parameters),
> how can I regard for the different p-values
> (which are calculated with different tests in the
> trials). Is it possible to form something like a
> weighet mean with the p-values and the sample
> sizes in the trials to generate an common effect
> size of the different results in the trials?
You might try some textbooks or articles concerning meta-analysis, and
read what you find online. There are some comments in my stats-FAQ.
You could look at examples of actual meta-analyses -- ones that were
not carried out by statisticians as models to be followed -- but be
aware that the majority of those are poorly done and misleading, both
as to style and conclusions.
There are two extremes in combining studies, and what you have
described is a confusion between the two. "p-level" heavily depends
on sample size (N) and sample characteristics. It is a measure of
outcome, "significant" or not, but not a measure of "effect".
A good indicator of "effect" is independent of sample -- using, for
instance, a raw measure like "years" or "inches" or "pounds." (And,
Odds ratios are sometimes independent, which is one reason that ORs
are increasing popular.) A measure like a correlation coefficient or
a standardized difference is independent of the sample N, but
*depends* on the sample characteristics -- so it is not very good for
blithe comparisons across *heterogeneous* samples.
If you have various studies that all might lean the same direction,
you can try to combine the p-levels in order to get an over-all
"significant" conclusion. There are a number of different formulas
for this, which provide different sorts of weights. Should one
extreme result count for more than several intermediate results?
Fisher's method emphasizes the former, other methods stress
consistency of moderate outcomes.
On the other hand, if you are sure that an effect is real but it needs
to be better described and articulated, then you can look at
characteristics of studies, and their samples, and the actual effect
that was obtained. Essentially, on performs an ANOVA on effect sizes,
after taking care that you have selected the studies well, and that
the outcomes are, after all, homogeneous enough to be combined.
How or whether to weight studies of different Ns is a perpetual
question. A bigger question might be, How or whether to discard
studies for being (a) irrelevant to the topic, or (b) incompetently
done.
Hope this helps.
--
Rich Ulrich, [EMAIL PROTECTED]
http://www.pitt.edu/~wpilib/index.html
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
