Recently I came across a similar issue, but I had a far greater number of repeated tests, 595 actually. If i used 10.000 permutations to access significance, I noticed that I could never reject the null hypothesis. That is because for 10.000 permutations, the minimum p value is 1e-04 and the bonferroni adjusted value is 0.0595. So, instead of using any multiple test corrections, I simply rejected the null-hypothesis when the observed statistic fell completely outside the distribution constructed by simulation. This produced nearly identical results to a parametric approach with bonferroni correction (I was analyzing correlations in that case).
I don’t know if that adds to the question initially raised, since for 18 tests, the minimum adjusted pvalues are still lower than 0.05, but this experience led me to believe that applying bonferroni correction to non-parametric p-value estimates is not that straightforward. I don’t know if another multiple-test correction method is more adequate for these cases. Any thoughts would be appreciated. best regards, Fabio Andrade Machado Laboratório de Evolução de Mamíferos Departamento de Genética e Biologia Evolutiva- USP f.mach...@usp.br <mailto:f.mach...@usp.br> ; macfa...@gmail.com <mailto:macfa...@gmail.com> +55 11 982631029 skype: fabio_a_machado Lattes: http://lattes.cnpq.br/3673327633303737 <http://lattes.cnpq.br/3673327633303737> Google Scholar: http://scholar.google.com/citations?hl=en&user=2l6-VrQAAAAJ <http://scholar.google.com/citations?hl=en&user=2l6-VrQAAAAJ> ResearchGate: https://www.researchgate.net/profile/Fabio_Machado2 <https://www.researchgate.net/profile/Fabio_Machado2> > On Dec 11, 2016, at 21:55, Theodore Garland <theodore.garl...@ucr.edu> wrote: > > Dear Ting-Wen, > > This is a question about statistical philosophy in general, not specific to > tests for phylogenetic signal. How best to correct for making multiple > tests with related data is a huge and complicated topic. Another issue is > whether your traits are correlated with each other, which would affect > views on what would be best to do. In any case, beware that > simple Bonferroni correction is probably overly conservative, so perhaps at > least try something like sequential Bonferroni correction, if you do > attempt correction. > > (Aside from the points above, I am assuming that the 18 compounds do not > add up to 100% of the sample. If they do, then you would only want to > analyze 17 of them.) > > Sincerely, > Ted Garland > > On Sun, Dec 11, 2016 at 2:06 PM, Chen, Ting-Wen < > ting-wen.c...@biologie.uni-goettingen.de> wrote: > >> Dear all, >> >> I’m analysing some chemical compositions of species and considering them >> as “traits”, let’s say, 18 different compounds concentration in 37 species. >> I test phylogenetic signals in the percentage concentration of these >> compounds using Blomberg’s K and Pagel’s lambda using the function >> “phylosig". In Blomberg’s K I apply randomisation for the traits values on >> the tree to have a p-value for the corresponding trait and in Pagel’s >> lambda using likelihood test to get the p-value, resulting in several >> traits with phylogenetic signals as indicated by both K and lambda. Because >> phylogenetic signal is tested one by one, i.e. repeating 18 times for 18 >> compounds. Would you suggest that I have to adjust the p-values using e.g. >> bonferroni corrections? I have some compounds with p-values for both K and >> lambda about 0.02 (e.g. compound “I", K=0.656, lambda=0.633), while some >> other about 0.002 (compound “R", K=0.849, lambda=0.817). Is it safe to >> conclude that compound “I” also has a phylogenetic signal? >> >> Any idea will be very appreciated. Thank you! >> >> All the best >> Ting-Wen >> >> -- >> Ting-Wen Chen >> J.F. Blumenbach Institute of Zoology and Anthropology >> Georg August University Goettingen >> Berliner Str. 28 >> D-37073 Goettingen, Germany >> Tel: +49-55139-10943 >> >> _______________________________________________ >> R-sig-phylo mailing list - R-sig-phylo@r-project.org >> https://stat.ethz.ch/mailman/listinfo/r-sig-phylo >> Searchable archive at http://www.mail-archive.com/r- >> sig-ph...@r-project.org/ > > [[alternative HTML version deleted]] > > _______________________________________________ > R-sig-phylo mailing list - R-sig-phylo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-phylo > Searchable archive at http://www.mail-archive.com/r-sig-phylo@r-project.org/ [[alternative HTML version deleted]] _______________________________________________ R-sig-phylo mailing list - R-sig-phylo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-phylo Searchable archive at http://www.mail-archive.com/r-sig-phylo@r-project.org/
