1/(B+1) is the significance level of the Monte Carlo test if the data give the most extreme value. (E.g. Ripley, 1987, p. 171.) This is a calculation, not a convention, and assumes a continuously distributed statistic.
On Mon, 8 Mar 2004, Spencer Graves wrote: > What is the standard convention for a Monte Carlo p-value when the > observed outcome is more extreme than all simulations? The example > provided by C�dric Fine produced a Monte Carlo p-value, according to > Kjetil Halvorsen, of "2.2e-16 (based on 2000 replicates)". This seems > inappropriate to me. > > By reading the code, I found that for this case, > > PVAL <- sum(tmp$results >= STATISTIC)/B > > where STATISTIC is the observed chi-square while "tmp$results" is a > vector of length B of chi-squares from Monte Carlo simulated tables with > the same marginals. Thus, PVAL ranges over seq(0, 1, length=(B+1)). > For the observed table, presumably, PVAL = 0. The function "chisq.test" > apparently returns an object of class "htest", and "print.htest" calls > "format.pval(x$p.value, digits = digits)", for which "format.pval(0, 4)" > is "< 2.2e-16". > > Can someone provide an appropriate reference or sense of the > literature on the appropriate number to report for a Monte Carlo p-value > when the observed is more extreme than all the simulations? A value of > 0 or "< 2.2e-16" violates my sense of the logic of this situation. If > the appropriate number is 0.5/B, then the line "PVAL <- sum(tmp$results > >= STATISTIC)/B" could be followed immediately by something like the > following: > > if(PVAL==0) PVAL <- 0.5/B > > Comments? > Best Wishes, > Spencer Graves > > [EMAIL PROTECTED] wrote: > > >On 8 Mar 2004 at 16:38, [EMAIL PROTECTED] wrote: > > > > > > > >>I do not manage to make a Fisher�s exact test with the next matrix : > >> > >> > >> > > > >You can consider using chisq.test() with the argument sim=TRUE: > > > > > > > >>mat <- matrix(scan(), 7, 12, byrow=TRUE) > >> > >> > >1: 1 3 0 1 2 9 0 0 2 5 8 6 > >13: 0 3 3 0 0 0 0 5 0 3 0 0 > >25: 0 0 0 0 0 10 0 0 0 0 10 0 > >37: 0 2 0 2 6 14 0 0 6 0 10 6 > >49: 0 5 0 0 0 7 0 0 0 2 8 0 > >61: 0 0 1 9 4 7 2 1 4 2 12 5 > >73: 0 6 0 0 0 0 0 0 0 5 1 3 > >85: > >Read 84 items > > > > > > > >>chisq.test(mat, sim=TRUE) > >> > >> > > > > Pearson's Chi-squared test with simulated p-value (based > > on 2000 replicates) > > > >data: mat > >X-squared = 218.3366, df = NA, p-value = < 2.2e-16 > > > >Kjetil Halvorsen > > > > > > > > > > > >> [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] > >>[1,] 1 3 0 1 2 9 0 0 2 5 8 6 > >>[2,] 0 3 3 0 0 0 0 5 0 3 0 0 > >>[3,] 0 0 0 0 0 10 0 0 0 0 10 0 > >>[4,] 0 2 0 2 6 14 0 0 6 0 10 6 > >>[5,] 0 5 0 0 0 7 0 0 0 2 8 0 > >>[6,] 0 0 1 9 4 7 2 1 4 2 12 5 > >>[7,] 0 6 0 0 0 0 0 0 0 5 1 3 > >> > >>but I do not understand why it does not work since I obtain the next > >>error message : > >> > >> > >> > >> > >> > >>>fisher.test(enfin.matrix) > >>> > >>> > >>Error in fisher.test(enfin.matrix) : Bug in FEXACT: gave negative key > >> > >>thank you for considering my application > >> > >>c�dric finet > >> > >>______________________________________________ > >>[EMAIL PROTECTED] mailing list > >>https://www.stat.math.ethz.ch/mailman/listinfo/r-help > >>PLEASE do read the posting guide! > >>http://www.R-project.org/posting-guide.html > >> > >> > > > >______________________________________________ > >[EMAIL PROTECTED] mailing list > >https://www.stat.math.ethz.ch/mailman/listinfo/r-help > >PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html > > > > > > ______________________________________________ > [EMAIL PROTECTED] mailing list > https://www.stat.math.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html > > -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595 ______________________________________________ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
