Just an additional note: what I find interesting (that is an euphemism) is that a paper such as that got published on 2006 when a whole bunch of detailed papers on the same topic had been published in the past. For instance, the first I pick from the pile is by J. Romano, "On the behavior of randomization tests without a group invariance assumption", JASA, 1990, 85 (411): 686-692. There are other related papers on that same issue of JASA. The relevance of same-variance assumption also shows up in permutation test textbooks (including, I think, Manly's, Good's, Noreen's, etc).
Best, R. On Wed, Apr 9, 2008 at 9:11 PM, Greg Snow <[EMAIL PROTECTED]> wrote: > A few comments, > > My first impression on reading that abstract was that it was complete > nonsense. After thinking a bit about it and skimming the full article I > decided that it was nonsense, but nonsense that is important to research and > discuss (and therefore the paper is useful). > > Why is it nonsense? The permutation test is a test of the null hypothesis > that the 2 (or k) groups are from the same distribution (or identically > distributed, or exchangable). The abstract says that they looked at the type > I error rate when the 2 groups had different variances or other differences. > The type I error is defined when the null hypothesis is true, so computing a > type I error rate when the null is by definition false does not make sense. > > However, statisticians often do analyses where all the assumptions are not > necessarily true (is any population really distributed as a normal), but the > tests are close enough. So with modern tools it is not suprising to see > people doing permutation tests without understanding what they are really > testing and the results may be close enough (or they might not be). The > contribution of this paper is to test and see if the results are close enough > or not when you use a permutation test to test the null that the means are > equal when there are other differences in the groups. Their answer is that > no, the results are not close enough and they suggest that if you want to > test for equality of means, but not identical distributions, then don't use a > permutation test. > > To expand on Thierry's original answer: > > If you are testing the correct hypotheses and doing a permutation test > correctly, then > "You can do permutation tests on an unbalanced design" and it will still be > a correct test. Unbalance could affect the power, which you would want to > take into account when designing a study, but does not affect the correctness > of the test (when used properly). > > Hope this helps, > > -- > Gregory (Greg) L. Snow Ph.D. > Statistical Data Center > Intermountain Healthcare > [EMAIL PROTECTED] > (801) 408-8111 > > > > > > -----Original Message----- > > From: [EMAIL PROTECTED] > > [mailto:[EMAIL PROTECTED] On Behalf Of João Fadista > > Sent: Tuesday, April 08, 2008 4:10 PM > > > > To: ONKELINX, Thierry; r-help@r-project.org > > Subject: Re: [R] permutation test assumption? > > > > Dear Thierry, > > > > Thanks for the reply. But as you may read in the paper > > http://bioinformatics.oxfordjournals.org/cgi/content/abstract/ > > 22/18/2244 when the sample sizes are not the same there may > > be an increase in the Type I error rate. > > > > Comments will be appreciated. > > > > Best regards, > > João Fadista > > > > > > ________________________________ > > > > De: ONKELINX, Thierry [mailto:[EMAIL PROTECTED] > > Enviada: ter 08-04-2008 15:27 > > Para: João Fadista; r-help@r-project.org > > Assunto: RE: [R] permutation test assumption? > > > > > > > > Dear João, > > > > You can do permutation tests on an unbalanced design. > > > > HTH, > > > > Thierry > > > > > > -------------------------------------------------------------- > > -------------- > > ir. Thierry Onkelinx > > Instituut voor natuur- en bosonderzoek / Research Institute > > for Nature and Forest Cel biometrie, methodologie en > > kwaliteitszorg / Section biometrics, methodology and quality > > assurance Gaverstraat 4 9500 Geraardsbergen Belgium tel. + 32 > > 54/436 185 [EMAIL PROTECTED] www.inbo.be > > > > To call in the statistician after the experiment is done may > > be no more than asking him to perform a post-mortem > > examination: he may be able to say what the experiment died of. > > ~ Sir Ronald Aylmer Fisher > > > > The plural of anecdote is not data. > > ~ Roger Brinner > > > > The combination of some data and an aching desire for an > > answer does not ensure that a reasonable answer can be > > extracted from a given body of data. > > ~ John Tukey > > > > -----Oorspronkelijk bericht----- > > Van: [EMAIL PROTECTED] > > [mailto:[EMAIL PROTECTED] Namens João Fadista > > Verzonden: dinsdag 8 april 2008 15:18 > > Aan: r-help@r-project.org > > Onderwerp: [R] permutation test assumption? > > > > Dear all, > > > > Can I do a permutation test if the number of individuals in > > one group is much bigger than in the other group? I searched > > the literature but I didin´t find any assumption that refers > > to this subject for permutation tests. > > > > > > Best regards > > > > João Fadista > > Ph.d. student > > > > > > > > UNIVERSITY OF AARHUS > > Faculty of Agricultural Sciences > > Dept. of Genetics and Biotechnology > > Blichers Allé 20, P.O. BOX 50 > > DK-8830 Tjele > > > > Phone: +45 8999 1900 > > Direct: +45 8999 1900 > > E-mail: [EMAIL PROTECTED] > > <mailto:[EMAIL PROTECTED]> > > Web: www.agrsci.org <http://www.agrsci.org/> > > ________________________________ > > > > DJF now offers new degree programmes > > <http://www.agrsci.org/content/view/full/34133> . > > > > News and news media > > <http://www.agrsci.org/navigation/nyheder_og_presse> . > > > > This email may contain information that is confidential. Any > > use or publication of this email without written permission > > from Faculty of Agricultural Sciences is not allowed. If you > > are not the intended recipient, please notify Faculty of > > Agricultural Sciences immediately and delete this email. > > > > > > > > > > [[alternative HTML version deleted]] > > > > ______________________________________________ > > R-help@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide > > http://www.R-project.org/posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. > > > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > -- Ramon Diaz-Uriarte Statistical Computing Team Structural Biology and Biocomputing Programme Spanish National Cancer Centre (CNIO) http://ligarto.org/rdiaz ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
