I can think of 2 better methods:
1. Bayes. Sample from the set of 100 trees, used as a prior on the
"true" tree structure (assuming they are a true posterior set, as Joe
points out). See: de Villemereuil et al. BMC Evolutionary Biology 2012,
12:102 http://www.biomedcentral.com/1471-2148/12/102
2. Model averaging using information theoretic methods. See package
MuMIn. From a Bayesian perspective, this is problematic as the Akaike
prior on each model depends on the data. But I don't think it is as
problematic as averaging p-values.
I don't like the methods that have previously been suggested. Think
about the definition of a p-value: The probability of obtaining a
statistic at least as extreme as that observed, conditional on the null
hypothesis being true. What is the null hypothesis in this case?
Probably that the estimate for a parameter is zero. For any single
analysis (say via PGLS), this has to be conditional on the tree
structure being correct! But you are using 100 different tree
structures. Perhaps only one is correct (if there are no duplicates),
and quite possibly none of them are correct. So trying to treat
p-values as some sort of variable that you can obtain summary statistics
for such as the median, mean, standard deviation etc. makes no sense
because each p-value is defined in terms of a different null hypothesis
for every analysis. This is different to when you might be testing the
same null hypothesis, but on different data sets. For example, you might
be trying to replicate a study from somebody else's lab (there should be
more of this.) Then the distribution of p-values from different data
sets should be Uniform under the null hypothesis. It is also different
from meta-analysis methods where p-values may be combined from sources
using different data.
HTH,
Simon.
On 11/03/16 05:35, Simon Joly wrote:
Alternatively, the proportion of trees that gave in a significant result
(for a given threshold) could be of interest. It depends on your question.
Simon
--------------------------------------------------------------------------------------------------------
Simon Joly, Ph.D.
Chercheur, Jardin botanique de Montréal / Espace pour la vie
Professeur associé, Dept. sciences biologiques, Université de Montréal
Institut de recherche en biologie végétale (IRBV)
4101 Sherbrooke E., Montréal (QC) H1X 2B2, Canada
T. +1 514.872.0344 / www.plantevolution.org
2016-03-10 11:41 GMT-05:00 David Bapst <dwba...@gmail.com>:
Darrin, list-
I'm sure there's people on this list with better answers, so I'll
throw in first with what might be the wrong answer (but feels right to
me), and say you more or less need to report all of them: like, show a
full histogram of the p-values. At least, as a reviewer, that is what
would convince me whether there was evidence or not to reject a
hypothesis.
But I'm sure there's some statistical argument again that too, in
terms of taking a frequentist perspective across multiple versions of
the same dataset.
To the list: I look forward to hearing how I am wrong! ;)
-Dave
On Thu, Mar 10, 2016 at 4:54 AM, Darrin Hulsey
<darrinhulseymin...@outlook.com> wrote:
I am running a series of statistics on a subset of 100 trees that
returns 100 different p-values. I was wondering what the best way to
report summary statistics for these 100 p-values would be (median?, measure
of variance in all 100 p-values?). Thanks for any insight.
[[alternative HTML version deleted]]
_______________________________________________
R-sig-phylo mailing list - R-sig-phylo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
Searchable archive at
http://www.mail-archive.com/r-sig-phylo@r-project.org/
--
David W. Bapst, PhD
Adjunct Asst. Professor, Geology and Geol. Eng.
South Dakota School of Mines and Technology
501 E. St. Joseph
Rapid City, SD 57701
http://webpages.sdsmt.edu/~dbapst/
http://cran.r-project.org/web/packages/paleotree/index.html
_______________________________________________
R-sig-phylo mailing list - R-sig-phylo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
Searchable archive at
http://www.mail-archive.com/r-sig-phylo@r-project.org/
[[alternative HTML version deleted]]
_______________________________________________
R-sig-phylo mailing list - R-sig-phylo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
Searchable archive at http://www.mail-archive.com/r-sig-phylo@r-project.org/
--
Simon Blomberg, BSc (Hons), PhD, MAppStat, AStat.
Senior Lecturer and Consultant Statistician
School of Biological Sciences
The University of Queensland
St. Lucia Queensland 4072
Australia
T: +61 7 3365 2506
email: S.Blomberg1_at_uq.edu.au
http://www.evolutionarystatistics.org
Policies:
1. I will NOT analyse your data for you.
2. Your deadline is your problem.
Basically, I'm not interested in doing research
and I never have been. I'm interested in
understanding, which is quite a different thing.
- David Blackwell
_______________________________________________
R-sig-phylo mailing list - R-sig-phylo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
Searchable archive at http://www.mail-archive.com/r-sig-phylo@r-project.org/
