Looks sound to me. I would think that
v =. (#t) {.!.B >: ".{.t
would suffice for v - the chance of all trailing 0s is small,
and the benefit of detecting it also small.
Henry Rich
On 5/21/2011 2:40 PM, Roger Hui wrote:
> Thanks. I realize that. After all, if ?1e6$12345
> results in z=:_1e6$(<.1e6%12345)$i.12345,
> z would satisfy the chi square test, KS test, etc.,
> but we would (should) be highly suspicious of
> the "RNG"!
>
> So I am depending on the fact that ?m generates
> uniform random numbers as well as having
> many other desirable properties. In this case it does,
> because ?m uses the Mersenne Twister.
>
>> For y an extended precision number, ?y is computed
>> by t=.?y1 where y1 is a little bigger than y,
>> repeating until t is less than y. (The same y1
>> is used for a given y.)
>
> In detail:
>
> B=: 10000x
> rand=: 3 : 0
> m=.<.10^.B
> t=. (-m) ]\ ((m|-#t)$' '),t=.":y
> n=.<:#t
> c=. ".{.t NB. leading digit
> v=. (c++./'0'~:,}.t),n#B
> whilst. y<:z do. z=. B#.?v end.
> )
>
> B is the base for extended integers. The loop
> repeatedly does z=.B#.?v until y>z . The
> number B#.B|v is the y1 that I described.
>
> The bug in the interpreter was that it was
> basically doing y<z instead of the correct y<:z .
>
> Examples:
>
> 11^20x
> 672749994932560009201
>
> rand 11^20x
> 554374486617845425729
> rand 11^20x
> 98359541759946207683
>
> t=: rand"0 ]1e5$12345x
> >./t
> 12344
> <./t
> 0
> (c%12344) , (+/c>:t)%#t [ c=: ?12345
> 0.225697 0.22524
>
>
>
> ----- Original Message -----
> From: Henry Rich<[email protected]>
> Date: Saturday, May 21, 2011 10:15
> Subject: Re: [Jprogramming] roll
> To: Programming forum<[email protected]>
>
>> Even if you know that the distribution of numbers from your RNG
>> is
>> uniform, that is not sufficient to ensure that there are not
>> pernicious
>> patterns in the numbers.
>>
>> For example: multiplicative congruence generators produce
>> sequences that
>> pass all sorts of tests, but it turns out that if you use sets
>> of n
>> sequential numbers produced by such an RNG to sample an n-
>> dimensional
>> space, the n-sets fall on hyperplanes of the n-dimensional
>> space,
>> leaving large areas unsampled.
>>
>> I don't know enough to offer any advice about picking an RNG for
>> general
>> use. My one experience with this sort of thing was when I
>> got some
>> surprising results using a multiplicative congruence RNG, and
>> replaced
>> it with Knuth's suggestion, which at the time was to use two
>> such RNGs,
>> shuffling the results from one using the results of the other,
>> and my
>> results became reasonable.
>>
>> Henry Rich
>>
>> On 5/21/2011 11:30 AM, Roger Hui wrote:
>>> Thanks for your reply. To paraphrase Ewart Shaw,
>>> my lack of depth in statistics is only ameliorated by
>>> the narrowness of my understanding. Regarding the
>>> last result:
>>>
>>>> -.14&chisqcdf f i.5
>>>> 0.262214 0.120479 0.124215 0.292923 0.68203
>>>
>>> What are the numbers saying? Do they indicate that the
>>> underlying distribution is nearly uniform or far from uniform?
>>> Is it a good or bad thing (or irrelevant) that the 5 numbers
>>> are quite different from each other?
>>>
>>> In "school" I learned that the Kolmogorov-Smirnov test
>>> http://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test
>>> is good for this kind of thing. What do you think of
>>> this test?
>>>
>>>
>>>
>>> ----- Original Message -----
>>> From: John Randall<[email protected]>
>>> Date: Saturday, May 21, 2011 7:52
>>> Subject: Re: [Jprogramming] roll
>>> To: Programming forum<[email protected]>
>>>
>>>> The process described should yield uniform random numbers
>>>> between 0
>>>> and y-1.
>>>>
>>>> Following up on the tests in the original posting, we can put the
>>>> numbers into equal sized bins and then calculate a chi-square
>>>> statistic. If this is not significant, we can be somewhat
>>>> assuredthat the numbers are random.
>>>>
>>>> f=:3 : 0"0
>>>> y=. 12345
>>>> t=. ?2e6$20000
>>>> z=. 1e6 {. (12345>t)#t
>>>> NB. 12345=15*823
>>>> bin=.<.@%&823
>>>> observed=.(bin z) #/. z
>>>> expected=.(#z)%15
>>>> chisquare=.+/(*:observed-expected)%expected
>>>> )
>>>>
>>>> require '~addons/stats/base/distribution.ijs'
>>>>
>>>> -.14&chisqcdf f i.5
>>>> 0.262214 0.120479 0.124215 0.292923 0.68203
>>>>
>>>> Best wishes,
>>>>
>>>> John
>>>>
>>>>
>>>>
>>>> Roger Hui wrote:
>>>>> I have found a bug in ?y where y is an extended
>>>>> precision integer. The result should never be y,
>>>>> but:
>>>>>
>>>>> t=: ? 1e6 $ 12345x
>>>>> +/t=12345
>>>>> 89
>>>>>
>>>>> The last result should of course be 0.
>>>>>
>>>>> I know how this can be fixed, but I would like to
>>>>> verify that the algorithm for ?y is correct with the
>>>>> mathematicians in this forum.
>>>>>
>>>>> For y an extended precision number, ?y is computed
>>>>> by t=.?y1 where y1 is a little bigger than y,
>>>>> repeating until t is less than y. (The same y1
>>>>> is used for a given y.) Assuming that ?m produces
>>>>> uniform random numbers between 0 and m-1, does
>>>>> this process give uniform random numbers between
>>>>> 0 and y-1?
>>>>>
>>>>> For example, suppose we want to compute z=:?1e6$y=:12345.
>>>>>
>>>>> y=: 12345
>>>>> t=: ?2e6$20000
>>>>> +/12345>t
>>>>> 1233158
>>>>> z=: 1e6 {. (12345>t)#t
>>>>>
>>>>> NB. various tests
>>>>> (+/z)%#z
>>>>> 6170.3
>>>>> 12344%2
>>>>> 6172
>>>>>
>>>>> (c%y) , (+/c>:z)%#z [ c=: ? y
>>>>> 0.993277 0.993333
>>>>> (c%y) , (+/c>:z)%#z [ c=: ? y
>>>>> 0.43872 0.438943
>>>>> (c%y) , (+/c>:z)%#z [ c=: ? y
>>>>> 0.593925 0.594519
>>>>> (c%y) , (+/c>:z)%#z [ c=: ? y
>>>>> 0.315674 0.315579
>
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
>
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
