Re: [R-sig-phylo] bonferroni corrections in Blomberg's K and Pagel's lambda

[Chen, Ting-Wen]

Hi Carmelo,

thank you so much for the suggestions. =)

All the best
Ting-Wen


--
Ting-Wen Chen
J.F. Blumenbach Institute of Zoology and Anthropology
Georg August University Goettingen
Berliner Str. 28
D-37073 Goettingen, Germany
Tel: +49-55139-10943

r-sig-phylo-requ...@r-project.org<mailto:r-sig-phylo-requ...@r-project.org> 於 
2016年12月16日 19:00 寫道：

Message: 5
Date: Fri, 16 Dec 2016 05:02:23 +0100
From: Carmelo Fruciano <c.fruci...@unict.it<mailto:c.fruci...@unict.it>>
To: r-sig-phylo@r-project.org<mailto:r-sig-phylo@r-project.org>
Subject: Re: [R-sig-phylo] bonferroni corrections in Blomberg's K and
Pagel's lambda
Message-ID: 
<20161216050223.47442dfb1b3g9...@webmail.unict.it<mailto:20161216050223.47442dfb1b3g9...@webmail.unict.it>>
Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed"

Hi Ting-Wen,
as Ted pointed out, if and how one has to correct for multiple tests
is a huge topic. Perhaps looking at the literature and making your own
opinion on this matter would be the best choice (for example, Perneger
1998 - British Medical Journal and Garcia 2004 - Oikos, present two
different points of view in a very accessible way).

A few extra points:
- There are other methods that are generally less conservative than
the Holm procedure ("Sequential Bonferroni"); these include the
Benjamini-Hochberg procedure (Benjamini & Hochberg 1995 - Journal of
the Royal Statistical Society; see also Benjamini & Yekutieli 2001 -
The Annals of Statistics) and other more recent procedures (such as
Carvajal-Rodriguez & de Una-Alvarez 2011 - Plos One); most of these
are worth checking out, making your opinion on them and, possibly,
using them (most of them have R implementations)

- If your dataset is multidimensional/multivariate in nature, you
might want to consider using multivariate approaches for estimating
and testing for phylogenetic signal (e.g., Adams 2014 - Systematic
Biology), rather than many univariate tests

- If your dataset/hypothesis is multivariate in nature, testing
individual PCs separately might be a poor choice (especially for PCs
of order higher than 1), if it is not multivariate in nature, there
might be no reason to use PCA

I hope this is of some help.
Best,
Carmelo



--
Carmelo Fruciano
Postdoctoral Fellow - Queensland University of Technology - Brisbane,
Australia
Honorary Fellow - University of Catania - Catania, Italy
e-mail c.fruci...@unict.it<mailto:c.fruci...@unict.it>
http://www.fruciano.it/research/


[[alternative HTML version deleted]]

___
R-sig-phylo mailing list - R-sig-phylo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
Searchable archive at http://www.mail-archive.com/r-sig-phylo@r-project.org/

Re: [R-sig-phylo] bonferroni corrections in Blomberg's K and Pagel's lambda

[Carmelo Fruciano]

Hi Ting-Wen,
as Ted pointed out, if and how one has to correct for multiple tests  
is a huge topic. Perhaps looking at the literature and making your own  
opinion on this matter would be the best choice (for example, Perneger  
1998 - British Medical Journal and Garcia 2004 - Oikos, present two  
different points of view in a very accessible way).


A few extra points:
- There are other methods that are generally less conservative than  
the Holm procedure ("Sequential Bonferroni"); these include the  
Benjamini-Hochberg procedure (Benjamini & Hochberg 1995 - Journal of  
the Royal Statistical Society; see also Benjamini & Yekutieli 2001 -  
The Annals of Statistics) and other more recent procedures (such as  
Carvajal-Rodriguez & de Una-Alvarez 2011 - Plos One); most of these  
are worth checking out, making your opinion on them and, possibly,  
using them (most of them have R implementations)


- If your dataset is multidimensional/multivariate in nature, you  
might want to consider using multivariate approaches for estimating  
and testing for phylogenetic signal (e.g., Adams 2014 - Systematic  
Biology), rather than many univariate tests


- If your dataset/hypothesis is multivariate in nature, testing  
individual PCs separately might be a poor choice (especially for PCs  
of order higher than 1), if it is not multivariate in nature, there  
might be no reason to use PCA


I hope this is of some help.
Best,
Carmelo



--
Carmelo Fruciano
Postdoctoral Fellow - Queensland University of Technology - Brisbane,  
Australia

Honorary Fellow - University of Catania - Catania, Italy
e-mail c.fruci...@unict.it
http://www.fruciano.it/research/



"Chen, Ting-Wen" <ting-wen.c...@biologie.uni-goettingen.de> ha scritto:


Dear Ted, dear Fabio,

thank you so much for your suggestions. I found that people applied  
bonferroni corrections in p-values in Pagel’s lambda, as shown in  
this paper: http://www.pnas.org/content/106/43/18097.abstract


In my case, I decide to use p.adjust (x,method=“holm”,n=length(x))  
to correct the p-values, hope this would be better. I also did some  
PCA to reduce the trait dimensions and to avoid trait correlation  
with each other, and then tested phylogenetic signal for the first  
several PCs.


All the best
Ting-Wen

--
Ting-Wen Chen
J.F. Blumenbach Institute of Zoology and Anthropology
Georg August University Goettingen
Berliner Str. 28
D-37073 Goettingen, Germany
Tel: +49-55139-10943

r-sig-phylo-requ...@r-project.org<mailto:r-sig-phylo-requ...@r-project.org>  
於 2016年12月13日 19:00 寫道：


Send R-sig-phylo mailing list submissions to
r-sig-phylo@r-project.org<mailto:r-sig-phylo@r-project.org>

To subscribe or unsubscribe via the World Wide Web, visit
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
or, via email, send a message with subject or body 'help' to
r-sig-phylo-requ...@r-project.org

You can reach the person managing the list at
r-sig-phylo-ow...@r-project.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of R-sig-phylo digest..."


Today's Topics:

  1. Re: bonferroni corrections in Blomberg's K and Pagel's lambda
 (F?bio Machado)


--

Message: 1
Date: Tue, 13 Dec 2016 08:49:59 -0200
From: F?bio Machado <macfa...@gmail.com>
To: "r-sig-phylo@r-project.org" <r-sig-phylo@r-project.org>
Subject: Re: [R-sig-phylo] bonferroni corrections in Blomberg's K and
Pagel's lambda
Message-ID: <bc3b8326-2692-44b9-94ae-56fc2d28b...@gmail.com>
Content-Type: text/plain; charset="UTF-8"

Recently I came across a similar issue, but I had a far greater  
number of repeated tests, 595 actually. If i used 10.000  
permutations to access significance, I noticed that I could never  
reject the null hypothesis. That is because for 10.000 permutations,  
the minimum p value is 1e-04 and the bonferroni adjusted value is  
0.0595. So, instead of using any multiple test corrections, I simply  
rejected the null-hypothesis when the observed statistic fell  
completely outside the distribution constructed by simulation. This  
produced nearly identical results to a parametric approach with  
bonferroni correction (I was analyzing correlations in that case).


I don?t know if that adds to the question initially raised, since  
for 18 tests, the minimum adjusted pvalues are still lower than  
0.05, but this experience led me to believe that applying bonferroni  
correction to non-parametric p-value estimates is not that  
straightforward. I don?t know if another multiple-test correction  
method is more adequate for these cases. Any thoughts would be  
appreciated.


best regards,

Fabio Andrade Machado
Laborat?rio de Evolu??o de Mam?feros
Departamento de Gen?tica e Biologia Evolutiva- USP
f.mach...@usp.br <mailto:f.mach...@usp.br> ; macfa...@gmail.com  
<mailto:macfa...@

Re: [R-sig-phylo] bonferroni corrections in Blomberg's K and Pagel's lambda

[Chen, Ting-Wen]

Dear Ted, dear Fabio,

thank you so much for your suggestions. I found that people applied bonferroni 
corrections in p-values in Pagel’s lambda, as shown in this paper: 
http://www.pnas.org/content/106/43/18097.abstract

In my case, I decide to use p.adjust (x,method=“holm”,n=length(x)) to correct 
the p-values, hope this would be better. I also did some PCA to reduce the 
trait dimensions and to avoid trait correlation with each other, and then 
tested phylogenetic signal for the first several PCs.

All the best
Ting-Wen

--
Ting-Wen Chen
J.F. Blumenbach Institute of Zoology and Anthropology
Georg August University Goettingen
Berliner Str. 28
D-37073 Goettingen, Germany
Tel: +49-55139-10943

r-sig-phylo-requ...@r-project.org<mailto:r-sig-phylo-requ...@r-project.org> 於 
2016年12月13日 19:00 寫道：

Send R-sig-phylo mailing list submissions to
r-sig-phylo@r-project.org<mailto:r-sig-phylo@r-project.org>

To subscribe or unsubscribe via the World Wide Web, visit
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
or, via email, send a message with subject or body 'help' to
r-sig-phylo-requ...@r-project.org

You can reach the person managing the list at
r-sig-phylo-ow...@r-project.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of R-sig-phylo digest..."


Today's Topics:

  1. Re: bonferroni corrections in Blomberg's K and Pagel's lambda
 (F?bio Machado)


--

Message: 1
Date: Tue, 13 Dec 2016 08:49:59 -0200
From: F?bio Machado <macfa...@gmail.com>
To: "r-sig-phylo@r-project.org" <r-sig-phylo@r-project.org>
Subject: Re: [R-sig-phylo] bonferroni corrections in Blomberg's K and
Pagel's lambda
Message-ID: <bc3b8326-2692-44b9-94ae-56fc2d28b...@gmail.com>
Content-Type: text/plain; charset="UTF-8"

Recently I came across a similar issue, but I had a far greater number of 
repeated tests, 595 actually. If i used 10.000 permutations to access 
significance, I noticed that I could never reject the null hypothesis. That is 
because for 10.000 permutations, the minimum p value is 1e-04 and the 
bonferroni adjusted value is 0.0595. So, instead of using any multiple test 
corrections, I simply rejected the null-hypothesis when the observed statistic 
fell completely outside the distribution constructed by simulation. This 
produced nearly identical results to a parametric approach with bonferroni 
correction (I was analyzing correlations in that case).

I don?t know if that adds to the question initially raised, since for 18 tests, 
the minimum adjusted pvalues are still lower than 0.05, but this experience led 
me to believe that applying bonferroni correction to non-parametric p-value 
estimates is not that straightforward. I don?t know if another multiple-test 
correction method is more adequate for these cases. Any thoughts would be 
appreciated.

best regards,

Fabio Andrade Machado
Laborat?rio de Evolu??o de Mam?feros
Departamento de Gen?tica e Biologia Evolutiva- USP
f.mach...@usp.br <mailto:f.mach...@usp.br> ; macfa...@gmail.com 
<mailto:macfa...@gmail.com>
+55 11 982631029
skype: fabio_a_machado

Lattes: http://lattes.cnpq.br/3673327633303737 
<http://lattes.cnpq.br/3673327633303737>
Google Scholar: http://scholar.google.com/citations?hl=en=2l6-VrQJ 
<http://scholar.google.com/citations?hl=en=2l6-VrQJ>
ResearchGate: https://www.researchgate.net/profile/Fabio_Machado2 
<https://www.researchgate.net/profile/Fabio_Machado2>
On Dec 11, 2016, at 21:55, Theodore Garland <theodore.garl...@ucr.edu> wrote:

Dear Ting-Wen,

This is a question about statistical philosophy in general, not specific to
tests for phylogenetic signal.  How best to correct for making multiple
tests with related data is a huge and complicated topic.  Another issue is
whether your traits are correlated with each other, which would affect
views on what would be best to do.  In any case, beware that
simple Bonferroni correction is probably overly conservative, so perhaps at
least try something like sequential Bonferroni correction, if you do
attempt correction.

(Aside from the points above, I am assuming that the 18 compounds do not
add up to 100% of the sample.  If they do, then you would only want to
analyze 17 of them.)

Sincerely,
Ted Garland

On Sun, Dec 11, 2016 at 2:06 PM, Chen, Ting-Wen <
ting-wen.c...@biologie.uni-goettingen.de> wrote:

Dear all,

I?m analysing some chemical compositions of species and considering them
as ?traits?, let?s say, 18 different compounds concentration in 37 species.
I test phylogenetic signals in the percentage concentration of these
compounds using Blomberg?s K and Pagel?s lambda using the function
?phylosig". In Blomberg?s K I apply randomisation for the traits values on
the tree to have a p-value for the corresponding trait and in Pagel?s
lambda using likelihood test to get the p-value, re

Re: [R-sig-phylo] bonferroni corrections in Blomberg's K and Pagel's lambda

[Fábio Machado]

Recently I came across a similar issue, but I had a far greater number of 
repeated tests, 595 actually. If i used 10.000 permutations to access 
significance, I noticed that I could never reject the null hypothesis. That is 
because for 10.000 permutations, the minimum p value is 1e-04 and the 
bonferroni adjusted value is 0.0595. So, instead of using any multiple test 
corrections, I simply rejected the null-hypothesis when the observed statistic 
fell completely outside the distribution constructed by simulation. This 
produced nearly identical results to a parametric approach with bonferroni 
correction (I was analyzing correlations in that case).

I don’t know if that adds to the question initially raised, since for 18 tests, 
the minimum adjusted pvalues are still lower than 0.05, but this experience led 
me to believe that applying bonferroni correction to non-parametric p-value 
estimates is not that straightforward. I don’t know if another multiple-test 
correction method is more adequate for these cases. Any thoughts would be 
appreciated.

best regards,

Fabio Andrade Machado
Laboratório de Evolução de Mamíferos 
Departamento de Genética e Biologia Evolutiva- USP
f.mach...@usp.br  ; macfa...@gmail.com 
 
+55 11 982631029
skype: fabio_a_machado

Lattes: http://lattes.cnpq.br/3673327633303737 

Google Scholar: http://scholar.google.com/citations?hl=en=2l6-VrQJ 

ResearchGate: https://www.researchgate.net/profile/Fabio_Machado2 

> On Dec 11, 2016, at 21:55, Theodore Garland  wrote:
> 
> Dear Ting-Wen,
> 
> This is a question about statistical philosophy in general, not specific to
> tests for phylogenetic signal.  How best to correct for making multiple
> tests with related data is a huge and complicated topic.  Another issue is
> whether your traits are correlated with each other, which would affect
> views on what would be best to do.  In any case, beware that
> simple Bonferroni correction is probably overly conservative, so perhaps at
> least try something like sequential Bonferroni correction, if you do
> attempt correction.
> 
> (Aside from the points above, I am assuming that the 18 compounds do not
> add up to 100% of the sample.  If they do, then you would only want to
> analyze 17 of them.)
> 
> Sincerely,
> Ted Garland
> 
> On Sun, Dec 11, 2016 at 2:06 PM, Chen, Ting-Wen <
> ting-wen.c...@biologie.uni-goettingen.de> wrote:
> 
>> Dear all,
>> 
>> I’m analysing some chemical compositions of species and considering them
>> as “traits”, let’s say, 18 different compounds concentration in 37 species.
>> I test phylogenetic signals in the percentage concentration of these
>> compounds using Blomberg’s K and Pagel’s lambda using the function
>> “phylosig". In Blomberg’s K I apply randomisation for the traits values on
>> the tree to have a p-value for the corresponding trait and in Pagel’s
>> lambda using likelihood test to get the p-value, resulting in several
>> traits with phylogenetic signals as indicated by both K and lambda. Because
>> phylogenetic signal is tested one by one, i.e. repeating 18 times for 18
>> compounds. Would you suggest that I have to adjust the p-values using e.g.
>> bonferroni corrections? I have some compounds with p-values for both K and
>> lambda about 0.02 (e.g. compound “I", K=0.656, lambda=0.633), while some
>> other about 0.002 (compound “R", K=0.849, lambda=0.817). Is it safe to
>> conclude that compound “I” also has a phylogenetic signal?
>> 
>> Any idea will be very appreciated. Thank you!
>> 
>> All the best
>> Ting-Wen
>> 
>> --
>> Ting-Wen Chen
>> J.F. Blumenbach Institute of Zoology and Anthropology
>> Georg August University Goettingen
>> Berliner Str. 28
>> D-37073 Goettingen, Germany
>> Tel: +49-55139-10943
>> 
>> ___
>> R-sig-phylo mailing list - R-sig-phylo@r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
>> Searchable archive at http://www.mail-archive.com/r-
>> sig-ph...@r-project.org/
> 
>   [[alternative HTML version deleted]]
> 
> ___
> R-sig-phylo mailing list - R-sig-phylo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
> Searchable archive at http://www.mail-archive.com/r-sig-phylo@r-project.org/


[[alternative HTML version deleted]]

___
R-sig-phylo mailing list - R-sig-phylo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
Searchable archive at http://www.mail-archive.com/r-sig-phylo@r-project.org/

Re: [R-sig-phylo] bonferroni corrections in Blomberg's K and Pagel's lambda

[Theodore Garland]

Dear Ting-Wen,

This is a question about statistical philosophy in general, not specific to
tests for phylogenetic signal.  How best to correct for making multiple
tests with related data is a huge and complicated topic.  Another issue is
whether your traits are correlated with each other, which would affect
views on what would be best to do.  In any case, beware that
simple Bonferroni correction is probably overly conservative, so perhaps at
least try something like sequential Bonferroni correction, if you do
attempt correction.

(Aside from the points above, I am assuming that the 18 compounds do not
add up to 100% of the sample.  If they do, then you would only want to
analyze 17 of them.)

Sincerely,
Ted Garland

On Sun, Dec 11, 2016 at 2:06 PM, Chen, Ting-Wen <
ting-wen.c...@biologie.uni-goettingen.de> wrote:

> Dear all,
>
> I’m analysing some chemical compositions of species and considering them
> as “traits”, let’s say, 18 different compounds concentration in 37 species.
> I test phylogenetic signals in the percentage concentration of these
> compounds using Blomberg’s K and Pagel’s lambda using the function
> “phylosig". In Blomberg’s K I apply randomisation for the traits values on
> the tree to have a p-value for the corresponding trait and in Pagel’s
> lambda using likelihood test to get the p-value, resulting in several
> traits with phylogenetic signals as indicated by both K and lambda. Because
> phylogenetic signal is tested one by one, i.e. repeating 18 times for 18
> compounds. Would you suggest that I have to adjust the p-values using e.g.
> bonferroni corrections? I have some compounds with p-values for both K and
> lambda about 0.02 (e.g. compound “I", K=0.656, lambda=0.633), while some
> other about 0.002 (compound “R", K=0.849, lambda=0.817). Is it safe to
> conclude that compound “I” also has a phylogenetic signal?
>
> Any idea will be very appreciated. Thank you!
>
> All the best
> Ting-Wen
>
> --
> Ting-Wen Chen
> J.F. Blumenbach Institute of Zoology and Anthropology
> Georg August University Goettingen
> Berliner Str. 28
> D-37073 Goettingen, Germany
> Tel: +49-55139-10943
>
> ___
> R-sig-phylo mailing list - R-sig-phylo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
> Searchable archive at http://www.mail-archive.com/r-
> sig-ph...@r-project.org/

[[alternative HTML version deleted]]

___
R-sig-phylo mailing list - R-sig-phylo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
Searchable archive at http://www.mail-archive.com/r-sig-phylo@r-project.org/