Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-06 Thread Duncan Murdoch
On 05/11/2017 10:58 AM, peter dalgaard wrote: On 5 Nov 2017, at 15:17 , Duncan Murdoch wrote: On 04/11/2017 10:20 PM, Daniel Nordlund wrote: Tirthankar, "random number generators" do not produce random numbers. Any given generator produces a fixed sequence of

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-06 Thread Serguei Sokol
Le 05/11/2017 à 15:17, Duncan Murdoch a écrit : On 04/11/2017 10:20 PM, Daniel Nordlund wrote: Tirthankar, "random number generators" do not produce random numbers.  Any given generator produces a fixed sequence of numbers that appear to meet various tests of randomness.  By picking a seed you

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-05 Thread Paul Gilbert
I'll point out that there is there is a large literature on generating pseudo random numbers for parallel processes, and it is not as easy as one (at least me) would intuitively think. By a contra-positive like thinking one might guess that it will not be easy to pick seeds in a way that will

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-05 Thread peter dalgaard
> On 5 Nov 2017, at 15:17 , Duncan Murdoch wrote: > > On 04/11/2017 10:20 PM, Daniel Nordlund wrote: >> Tirthankar, >> "random number generators" do not produce random numbers. Any given >> generator produces a fixed sequence of numbers that appear to meet >> various

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-05 Thread Tirthankar Chakravarty
Duncan, Daniel, Thanks and indeed we intend to take the advice that Radford and Lukas have provided in this thread. I do want to re-iterate that the generating system itself cannot have any conception of the use of form IDs as seeds for a PRNG *and* the system itself only generates a sequence of

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-05 Thread Duncan Murdoch
On 04/11/2017 10:20 PM, Daniel Nordlund wrote: Tirthankar, "random number generators" do not produce random numbers. Any given generator produces a fixed sequence of numbers that appear to meet various tests of randomness. By picking a seed you enter that sequence in a particular place and

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-05 Thread Daniel Nordlund
Tirthankar, "random number generators" do not produce random numbers. Any given generator produces a fixed sequence of numbers that appear to meet various tests of randomness. By picking a seed you enter that sequence in a particular place and subsequent numbers in the sequence appear to

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-04 Thread Radford Neal
In the code below, you seem to be essentially using the random number generator to implement a hash function. This isn't a good idea. My impression is that pseudo-random number generation methods are generally evaluated by whether the sequence produced from any seed "appears" to be random.

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-03 Thread Tirthankar Chakravarty
Bill, Appreciate the point that both you and Serguei are making, but the sequence in question is not a selected or filtered set. These are values as observed in a sequence from a mechanism described below. The probabilities required to generate this exact sequence in the wild seem staggering to

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-03 Thread William Dunlap via R-devel
Another other generator is subject to the same problem with the same probabilitiy. > Filter(function(s){set.seed(s, kind="Knuth-TAOCP-2002");runif(1,17,26)>25.99}, 1:1) [1] 280 415 826 1372 2224 2544 3270 3594 3809 4116 4236 5018 5692 7043 7212 7364 7747 9256 9491 9568 9886 Bill Dunlap

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-03 Thread Tirthankar Chakravarty
Bill, I have clarified this on SO, and I will copy that clarification in here: "Sure, we tested them on other 8-digit numbers as well & we could not replicate. However, these are honest-to-goodness numbers generated by a non-adversarial system that has no conception of these numbers being used

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-03 Thread William Dunlap via R-devel
The random numbers in a stream initialized with one seed should have about the desired distribution. You don't win by changing the seed all the time. Your seeds caused the first numbers of a bunch of streams to be about the same, but the second and subsequent entries in each stream do look

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-03 Thread Serguei Sokol
Le 03/11/2017 à 14:24, Tirthankar Chakravarty a écrit : Martin, Thanks for the helpful reply. Alas I had forgotten that (implied) unfavorable comparisons of *nix systems with Windows systems would likely draw irate (but always substantive) responses on the R-devel list -- poor phrasing on my

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-03 Thread Tirthankar Chakravarty
Martin, Thanks for the helpful reply. Alas I had forgotten that (implied) unfavorable comparisons of *nix systems with Windows systems would likely draw irate (but always substantive) responses on the R-devel list -- poor phrasing on my part. :) Regardless, let me try to address some of the

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-03 Thread Lukas Stadler
If I interpret the original message as “I think there’s something wrong with R's random number generator”: Your assumption is that going from the seed to the first random number is a good hash function, which it isn’t. E.g., with Mersenne Twister it’s a couple of multiplications, bit shifts,

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-03 Thread Martin Maechler
> Tirthankar Chakravarty > on Fri, 3 Nov 2017 13:19:12 +0530 writes: > This is cross-posted from SO > (https://stackoverflow.com/q/47079702/1414455), but I now > feel that this needs someone from R-Devel to help > understand why this is

[Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

2017-11-03 Thread Tirthankar Chakravarty
This is cross-posted from SO (https://stackoverflow.com/q/47079702/1414455), but I now feel that this needs someone from R-Devel to help understand why this is happening. We are facing a weird situation in our code when using R's [`runif`][1] and setting seed with `set.seed` with the `kind =