Earlier today I reported finding an unbalanced number of observations in the p=0.0001 tails of rnorm.
Many thanks to Peter Dalgaard who suggested changing the normal.kind generator. Using RNGkind(kind = NULL, normal.kind ="Box-Muller") seems to have provided the remedy. For example: > observed.fraction.below [1] 0.000103 > observed.fraction.above [1] 0.000101 > Thank you, Peter! Charles Annis, P.E. [EMAIL PROTECTED] phone: 561-352-9699 eFAX: 503-217-5849 http://www.StatisticalEngineering.com -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Peter Dalgaard BSA Sent: Tuesday, January 28, 2003 2:36 PM To: Charles Annis, P.E. Cc: [EMAIL PROTECTED] Subject: Re: [R] random number generator? "Charles Annis, P.E." <[EMAIL PROTECTED]> writes: > Dear R-Aficionados: > > I realize that no random number generator is perfect, so what I report > below may be a result of that simple fact. However, if I have made an > error in my thinking I would greatly appreciate being corrected. > > I wish to illustrate the behavior of small samples (n=10) and so > generate 100,000 of them. > > n.samples <- 1000000 > sample.size = 10 > p <- 0.0001 > z.normal <- qnorm(p) > # generate n.samples of sample.size each from a normal(mean=0, sd=1) > density > # > small.sample <- matrix(rnorm(n=sample.size*n.samples, mean=0, sd=1), > nrow=n.samples, ncol=sample.size) > # Verify that from the entire small.sample matrix, p sampled values are > below, p above. > # > observed.fraction.below <- sum(small.sample < > z.normal)/length(small.sample) > observed.fraction.above <- sum(small.sample > > -z.normal)/length(small.sample) > > > observed.fraction.below > [1] 6.3e-05 > > observed.fraction.above > [1] 0.000142 > > > > I've checked the behavior of the entire sample's mean and median and > they seem fine. The total fraction in both tails is 0.0002, as it > should be. However in every instance about 1/3 are in the lower tail, > 2/3 in the upper. I also observe the same 1/3:2/3 ratio for one million > samples of ten. > > Is this simply because random number generators aren't perfect? Or have > I stepped in something? > > Thank you for your kind counsel. You stepped in something, I think, but I probably shouldn't elaborate on the metaphor ... There's an unfortunate interaction between the two methods that are used for generating uniform and normal variables (the latter uses the former). This has been reported a couple of times before and typically gives anomalous tail behaviour. Changing one of the generators (see help(RNGkind)) usually helps. -- O__ ---- Peter Dalgaard Blegdamsvej 3 c/ /'_ --- Dept. of Biostatistics 2200 Cph. N (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~~~~~~~~~ - ([EMAIL PROTECTED]) FAX: (+45) 35327907 ______________________________________________ [EMAIL PROTECTED] mailing list http://www.stat.math.ethz.ch/mailman/listinfo/r-help ______________________________________________ [EMAIL PROTECTED] mailing list http://www.stat.math.ethz.ch/mailman/listinfo/r-help
