Re: [R-sig-eco] 2-way adonis (PERMANOVA) incl interaction - how to test for main effects?

[Jari Oksanen]

Gian,

The marginal tests for one-term models had a problem in vegan 2.5-1 and vegan 
2.5-2. This is fixed in vegan 2.5-3, so please upgrade. (It never occurred to 
me that people would use marginal tests for one-term models where the fitted 
model is compared against the empty model, and that case was not handled 
correctly: any other test — total or sequential — should yield identical 
results in this case and marginal tests make no sense).

I haven’t even considered implementing type-XXX tests in vegan::adonis. The 
function uses standard R stats functions which only allow sequential tests or 
tests of marginal terms (ignoring non-marginal main effects when there are 
interactions). However, if you are interested in interaction term A:B, your 
really should inspect the model A+B+A:B. It makes no sense in any sense to 
study model A:B without the main effects. That would give misleading results 
for A:B. The interpretation of ~A:B is based on standard R 
stats::model.matrix() in vegan, and these seem to expand a bit differently for 
factors A & B and continuous variables A & B. So beware, brother beware. Use 
stats::model.matrix() to see how they really unfold in your case.

Cheers, Giari

On 1 Nov 2018, at 20:10, Gian Maria Niccolò Benucci 
mailto:gian.benu...@gmail.com>> wrote:

Thank you very much Steve,

I really appreciate you answer. I do not want to drop the main effects out
form the model, I was just trying to understand how adonis works. I was
more confused on how to use the adonis function correctly rather than on
the theory that is behind it. I have an almost "fully crossed" design (that
was the plan, the fact is that I have to remove 2 samples because of the
poor sequencing results) with two factors "Growhouse" and "Stage" in this
case. I am of course interested in the effect of the main factors, but I
also wanted to see if there was a significant interaction and if my
developmental stage (i.e. mature or young) was influenced by where my
organism where growing (i.e. growhouse 1 or 2).

Iif I do as you said, Type I SS for ~ A + B + A*B depends on order so... it
is sequential SS(A) SS(B|A) SS(A*B|A B)

The interaction is not significant, so maybe worth in this case perform a
Type II test directly?

adonis(t(otu_fungi_out) ~ Stage + Growhouse + Stage : Growhouse,
data=metadata_fungi_out, permutations=)

Call:
adonis(formula = t(otu_fungi_out) ~ Stage + Growhouse +
Stage:Growhouse,  data = metadata_fungi_out, permutations = )

Permutation: free
Number of permutations: 

Terms added sequentially (first to last)

   Df SumsOfSqs MeanSqs F.Model  R2 Pr(>F)
Stage10.4877 0.48769  2.7060 0.10408 0.0238 *
Growhouse10.3765 0.37647  2.0889 0.08034 0.0561 .
Stage:Growhouse  10.2171 0.21708  1.2045 0.04633 0.2376
Residuals   203.6045 0.18023 0.76925
Total   234.6857 1.0
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

adonis(t(otu_fungi_out) ~ Growhouse + Stage + Stage : Growhouse,
data=metadata_fungi_out, permutations=)

Call:
adonis(formula = t(otu_fungi_out) ~ Growhouse + Stage +
Stage:Growhouse,  data = metadata_fungi_out, permutations = )

Permutation: free
Number of permutations: 

Terms added sequentially (first to last)

   Df SumsOfSqs MeanSqs F.Model  R2 Pr(>F)
Growhouse10.3765 0.37647  2.0889 0.08034 0.0563 .
Stage10.4877 0.48769  2.7060 0.10408 0.0271 *
Growhouse:Stage  10.2171 0.21708  1.2045 0.04633 0.2364
Residuals   203.6045 0.18023 0.76925
Total   234.6857 1.0
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Thank you,

Gian




On Thu, 1 Nov 2018 at 11:06, Steve Brewer 
mailto:jbre...@olemiss.edu>> wrote:

Gian,

I am bit confused by what your concern is. First, if the imbalance is not
that severe, the approach you take to analyzing a two-way permanova (type
I, type II, type III ss) is not going to matter that much. Indeed, if the
design were balanced, they would give you identical results. Second,
regardless of the lack of balance, for the models y ~ A + B + A:B  and y ~
B + A + A:B, the test for the interaction will be the same. So, I don’t
understand why you would want to drop the main effects from the model,
effectively combining them with interaction. That doesn’t make any sense to
me. The problem is with the tests of the main effects.

My advice is to run both models (i.e., A first, then B first) using type I
ss. As mentioned, both models will give you the same interaction result. If
the interaction is all that you’re interested in, problem solved. Interpret
only the interaction and and ignore the main effects. If the interaction is
not significant and low, then interpret only the main effects, focusing
only on the second main effect in each of the differently-ordered models
(which are equivalent to Type II ss tests). And 

Re: [R-sig-eco] 2-way adonis (PERMANOVA) incl interaction - how to test for main effects?

[Gian Maria Niccolò Benucci]

Thank you very much Steve,

I really appreciate you answer. I do not want to drop the main effects out
form the model, I was just trying to understand how adonis works. I was
more confused on how to use the adonis function correctly rather than on
the theory that is behind it. I have an almost "fully crossed" design (that
was the plan, the fact is that I have to remove 2 samples because of the
poor sequencing results) with two factors "Growhouse" and "Stage" in this
case. I am of course interested in the effect of the main factors, but I
also wanted to see if there was a significant interaction and if my
developmental stage (i.e. mature or young) was influenced by where my
organism where growing (i.e. growhouse 1 or 2).

Iif I do as you said, Type I SS for ~ A + B + A*B depends on order so... it
is sequential SS(A) SS(B|A) SS(A*B|A B)

The interaction is not significant, so maybe worth in this case perform a
Type II test directly?

> adonis(t(otu_fungi_out) ~ Stage + Growhouse + Stage : Growhouse,
data=metadata_fungi_out, permutations=)

Call:
adonis(formula = t(otu_fungi_out) ~ Stage + Growhouse +
Stage:Growhouse,  data = metadata_fungi_out, permutations = )

Permutation: free
Number of permutations: 

Terms added sequentially (first to last)

Df SumsOfSqs MeanSqs F.Model  R2 Pr(>F)
Stage10.4877 0.48769  2.7060 0.10408 0.0238 *
Growhouse10.3765 0.37647  2.0889 0.08034 0.0561 .
Stage:Growhouse  10.2171 0.21708  1.2045 0.04633 0.2376
Residuals   203.6045 0.18023 0.76925
Total   234.6857 1.0
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

> adonis(t(otu_fungi_out) ~ Growhouse + Stage + Stage : Growhouse,
data=metadata_fungi_out, permutations=)

Call:
adonis(formula = t(otu_fungi_out) ~ Growhouse + Stage +
Stage:Growhouse,  data = metadata_fungi_out, permutations = )

Permutation: free
Number of permutations: 

Terms added sequentially (first to last)

Df SumsOfSqs MeanSqs F.Model  R2 Pr(>F)
Growhouse10.3765 0.37647  2.0889 0.08034 0.0563 .
Stage10.4877 0.48769  2.7060 0.10408 0.0271 *
Growhouse:Stage  10.2171 0.21708  1.2045 0.04633 0.2364
Residuals   203.6045 0.18023 0.76925
Total   234.6857 1.0
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Thank you,

Gian




On Thu, 1 Nov 2018 at 11:06, Steve Brewer  wrote:

> Gian,
>
> I am bit confused by what your concern is. First, if the imbalance is not
> that severe, the approach you take to analyzing a two-way permanova (type
> I, type II, type III ss) is not going to matter that much. Indeed, if the
> design were balanced, they would give you identical results. Second,
> regardless of the lack of balance, for the models y ~ A + B + A:B  and y ~
> B + A + A:B, the test for the interaction will be the same. So, I don’t
> understand why you would want to drop the main effects from the model,
> effectively combining them with interaction. That doesn’t make any sense to
> me. The problem is with the tests of the main effects.
>
> My advice is to run both models (i.e., A first, then B first) using type I
> ss. As mentioned, both models will give you the same interaction result. If
> the interaction is all that you’re interested in, problem solved. Interpret
> only the interaction and and ignore the main effects. If the interaction is
> not significant and low, then interpret only the main effects, focusing
> only on the second main effect in each of the differently-ordered models
> (which are equivalent to Type II ss tests). And these results will tell you
> pretty the same thing as type III tests if there is little or no
> interaction. I would not worry about trying to estimate the main effects
> while controlling for the interaction (Ellen’s question), which cannot be
> done using type I or type II SS in 2-way permanova using adonis. But why
> would you want to? The lack of a balanced design results in the main
> effects and the interaction not being independent of one another.  Forcing
> that independence by using type III ss can only work by essentially
> "throwing away" some of the information associated with the main effects,
> possibly resulting in an overly conservative test. The lower the
> interaction, however, the less is thrown away and the less it matters.
>
>
> Steve
>
>
>
>
> Stephen Brewer
> jbre...@olemiss.edu
> Professor
> Department of Biology
> PO Box 1848
>  University of Mississippi
> University, Mississippi 38677-1848
>  Brewer web page - https://jstephenbrewer.wordpress.com
> FAX - 662-915-5144 Phone - 662-202-5877
>
>
>
>
>
> On Oct 31, 2018, at 5:45 PM, Gian Maria Niccolò Benucci <
> gian.benu...@gmail.com> wrote:
>
> Thank you Jari,
>
> So to test if there are significant interaction I should use
> Stage:Growhouse
> i.e. A:B. This will test the interaction and main 

Re: [R-sig-eco] 2-way adonis (PERMANOVA) incl interaction - how to test for main effects?

[Steve Brewer]

Gian,

I am bit confused by what your concern is. First, if the imbalance is not that 
severe, the approach you take to analyzing a two-way permanova (type I, type 
II, type III ss) is not going to matter that much. Indeed, if the design were 
balanced, they would give you identical results. Second, regardless of the lack 
of balance, for the models y ~ A + B + A:B  and y ~ B + A + A:B, the test for 
the interaction will be the same. So, I don’t understand why you would want to 
drop the main effects from the model, effectively combining them with 
interaction. That doesn’t make any sense to me. The problem is with the tests 
of the main effects.

My advice is to run both models (i.e., A first, then B first) using type I ss. 
As mentioned, both models will give you the same interaction result. If the 
interaction is all that you’re interested in, problem solved. Interpret only 
the interaction and and ignore the main effects. If the interaction is not 
significant and low, then interpret only the main effects, focusing only on the 
second main effect in each of the differently-ordered models (which are 
equivalent to Type II ss tests). And these results will tell you pretty the 
same thing as type III tests if there is little or no interaction. I would not 
worry about trying to estimate the main effects while controlling for the 
interaction (Ellen’s question), which cannot be done using type I or type II SS 
in 2-way permanova using adonis. But why would you want to? The lack of a 
balanced design results in the main effects and the interaction not being 
independent of one another.  Forcing that independence by using type III ss can 
only work by essentially "throwing away" some of the information associated 
with the main effects, possibly resulting in an overly conservative test. The 
lower the interaction, however, the less is thrown away and the less it matters.


Steve




Stephen Brewer
jbre...@olemiss.edu
Professor
Department of Biology
PO Box 1848
 University of Mississippi
University, Mississippi 38677-1848
 Brewer web page - https://jstephenbrewer.wordpress.com
FAX - 662-915-5144 Phone - 662-202-5877





On Oct 31, 2018, at 5:45 PM, Gian Maria Niccolò Benucci 
mailto:gian.benu...@gmail.com>> wrote:

Thank you Jari,

So to test if there are significant interaction I should use Stage:Growhouse
i.e. A:B. This will test the interaction and main effects that are marginal
and so removed. How matters then if I include by="margin" or not? The R2
are the same (please see below) but the p-value changes. I assume the
second way is most correct, is it?

*> adonis2(t(otu_fungi_out) ~ Stage : Growhouse, data=metadata_fungi_out,
permutations=)*
Permutation test for adonis under reduced model
Terms added sequentially (first to last)
Permutation: free
Number of permutations: 

adonis2(formula = t(otu_fungi_out) ~ Stage:Growhouse, data =
metadata_fungi_out, permutations = )
   Df SumOfSqs  R2  F Pr(>F)
Stage:Growhouse  3   1.0812 0.23075 1.9998 0.0211 *
Residual20   3.6045 0.76925
Total   23   4.6857 1.0
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


*> adonis2(t(otu_fungi_out) ~ Stage : Growhouse, data=metadata_fungi_out,
by = "margin", permutations=)*
Permutation test for adonis under reduced model
Marginal effects of terms
Permutation: free
Number of permutations: 

adonis2(formula = t(otu_fungi_out) ~ Stage:Growhouse, data =
metadata_fungi_out, permutations = , by = "margin")
   Df SumOfSqs  R2  F Pr(>F)
Stage:Growhouse  3   1.0812 0.23075 1.9998  0.006 **
Residual20   3.6045 0.76925
Total   23   4.6857 1.0
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


Cheers,

Gian

On Tue, 30 Oct 2018 at 05:47, Jari Oksanen 
mailto:jari.oksa...@oulu.fi>> wrote:

Hello Gian,

These formulae expand into different models. Compare

model.matrix(~ Stage:Growhouse, data=metadata_fungi_out)
model.matrix(~ Stage*Growhouse, data=metadata_fungi_out)

The first model (Stage:Growhouse) will also contain (implicitly) main
effects and all these terms are marginal and can be removed, whereas the
latter Stage*Growhouse expands to explicit main effects and interaction
effects, and only the interaction effects are marginal and can be removed.
This is also reflected in the degrees of freedom in your anova table: In
the first case Stage:Growhouse has 3 df, and in the latter only 1 df (and
the main effects ignored had 2 df).

Ciao, Giari

On 29 Oct 2018, at 19:11, Gian Maria Niccolò Benucci <
gian.benu...@gmail.com> wrote:

Hello Jari,

It is a little bit confusing. If A*B unfolds in A+B+A:B then A:B is the
real interaction component.
So, which if the code below will test the variance for the interaction
alone?

adonis2(t(otu_fungi_out) ~ *Stage : Growhouse*, data=metadata_fungi_out,
by = "margin", permutations=)
Permutation test