Hello Mike, Thank you for your helpful reply. Interesting as indeed the landmark density within the blocks of my posted test examples are not huge: ranging from 29 to 13. I had kept these low being cautious about obtaining inflated r-pls values due to sample size/number of landmarks. Out of curiosity, how few where the few in your test when you saw proper numeric divergence in the effect values?
Maybe helpful to know: I have run the same test with the modularity.test function, and both sets obtained the exact same results: CR value of 0.7357 (p=0.001), with effect size of -8.4022. So it does seem to be, or at least in my case, confined to multi-block PLS effects. With best wishes and many thanks, K. > On 19 Oct 2020, at 17:57, Mike Collyer <[email protected]> wrote: > > Katrien, > > Changing the order of factor levels can certainly change Z-scores and > P-values, but I was surprised that it changed as much as you observed. The > reason it is not constant is because the permutation schedule (random > assignment of rows of values) will change with rearrangement of the module > blocks. However, I would expect the differences to be subtle. > > I was able to recreate something like your scenario (using integration.test) > with simulated data but found it only occurred if I had few landmarks per > module, suggesting this could be a landmark density issue. Does that seem > consistent with your data (are there only a few landmarks in each of the five > modules)? When I increased the number of landmarks, I got what I expected: > different but qualitatively similar Z-scores and P-values. > > Regarding the comapre.pls approach, the pooled standard error is calculated > differently, so a > change in this case should be expected. > > Hope that is helpful. > > Mike > > >> On Oct 19, 2020, at 9:55 AM, Katrien Janin <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hey all, >> >> I hope somebody can help me out to make sense of the following: >> Example #1 and #2 draw upon the same data set and same partition, just that >> the blocks are in reverse order. >> >> #1 > L.sym.nh.int5.pairwise >> A B C D E >> A 0.000 0.782 0.688 0.756 0.791 >> B 0.782 0.000 0.750 0.879 0.679 >> C 0.688 0.750 0.000 0.723 0.815 >> D 0.756 0.879 0.723 0.000 0.632 >> E 0.791 0.679 0.815 0.632 0.000 >> >> #2 > L.sym.nh.int5.pairwise >> E D C B A >> E 0.000 0.632 0.815 0.679 0.791 >> D 0.632 0.000 0.723 0.879 0.756 >> C 0.815 0.723 0.000 0.750 0.688 >> B 0.679 0.879 0.750 0.000 0.782 >> A 0.791 0.756 0.688 0.782 0.000 >> >> both obtain an overall r-PLS: 0.750 (p= 0.001), and as you can see in the >> tables the individual pairs obtain the exact same r-pls (as expected). >> >> the odd thing is that their effect size differ: example #1 returns an effect >> size of 2.6798 whilst #2 has a effect size of 4.640. >> >> any ideas what may be causing this? what can I do to find out what creates >> this difference in effect size, and which one I can report along the r-pls >> values? >> >> >> K. >> >> PS. I also computed the pls and effect for each individual pair with >> two.block: I yet again obtain the same r-pls values as in tables above (as >> expected) and when I average the effects of the pairs as obtained with the >> compare function I arrive at 5.626 ... >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Morphmet" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/morphmet2/4bb1e5d3-541e-4ce6-a9b0-aefd357d4a7fn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/morphmet2/4bb1e5d3-541e-4ce6-a9b0-aefd357d4a7fn%40googlegroups.com?utm_medium=email&utm_source=footer>. > -- You received this message because you are subscribed to the Google Groups "Morphmet" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/morphmet2/8CEFB831-241F-4CAF-8044-256B4451B658%40gmail.com.
