Dear Rebeka, The RRPP::pairwise function does not adjust P-values. It reports the P-values as a percentile of the observed values in the distributions of random values from the permutation procedure. One can adjust their pairwise level of significance if they wish — there is an option for that to illustrate the critical value in the distribution — but there is no attempt to “correct” P-values based on a particular philosophy.
Furthermore, one should be careful about correcting P-values that have a finite limit based on the arbitrary number of permutations performed. For example, with 1,000 random permutations a P-value of 0.001 is the lower limit. For 10,000, 0.0001 is the lower limit. One could make a judgement about “significance”, influenced perhaps by using fewer random permutations than would be required to find significance. As for whether P-value corrections are necessary for statistics with distributions generated from a resampling procedure, I have never had a strong opinion whether they should or should not be performed, especially when you are generating 15 pairwise differences between means, presumably among 6 means, in every random permutation of the data, for the same data. (There might be some programs or functions that generate 15 separate distributions from 15 different permutation procedures of 15 different data sets, maybe by subsetting the data by particular groups.) Because resampling procedures are not universally the same, it is difficult to have a blanket opinion about what pairwise significance levels should be. Perhaps worrying less about an arbitrary criterion and focusing more on the effect sizes from the tests is a better way to go. Best, Mike > On Nov 29, 2021, at 3:11 PM, Rebeka Rmoutilová <[email protected]> wrote: > > Hi! > > I have shape data with groups by sex and age (young, middle and old). I used > procSym function to perform Procrustes ANOVA on the data and now I want to > use a post-hoc test to find which groups are different. Totally, there is 15 > combinations of sex and age groups, but I am interested only in some of the > comparisons to find whether there is sexual dimorphism inside the age groups > or whether the age groups differ within males or females. So for example, I > am not interested whether there is a difference between young males and old > females which leads to only 9 comparisons. > > There are two functions I could probably use: permudist in Morpho package and > pairwise in RRPP package. Both functions compute differences between all > possible pairs of groups applying a stricter p value correction than if I > make only the 9 comparisons. Can I test the differences with one of those > functions without any p value correction and then correct it for 9 > comparisons using the p.adjust function? I see that at least in the permudist > function, it is possible to use no adjustment method, but I am not sure > whether it is a correct approach. Additionally, is it necessary to use p > value correction when I use permutations in the permudist function? > > Rebeka > > -- > You received this message because you are subscribed to the Google Groups > "Morphmet" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/morphmet2/8450baec-0ec3-4b1f-be42-ed6fef8ab811n%40googlegroups.com > > <https://groups.google.com/d/msgid/morphmet2/8450baec-0ec3-4b1f-be42-ed6fef8ab811n%40googlegroups.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "Morphmet" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/morphmet2/8A7D3FEF-812C-4E6E-9C92-AB516FBC4FE0%40gmail.com.
