[
https://issues.apache.org/jira/browse/RNG-122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16945809#comment-16945809
]
Alex Herbert commented on RNG-122:
----------------------------------
There are new long period {{XoRoShiRo}} generators by Vigna and Blackman that
are replacements for {{XOR_SHIFT_1024_S}}.
I have tested the lower and upper bits in both forward and reverse directions
for 10 runs of BigCrush:
h3. Lower 32-bits
{noformat}
RNG Bit-reversed TestU01 (BigCrush)
XO_RO_SHI_RO_1024_PP false 0,0,0,1,0,0,0,0,1,0
XO_RO_SHI_RO_1024_PP true 0,0,0,0,0,0,0,0,0,0
XO_RO_SHI_RO_1024_S false 0,0,0,0,2,0,0,0,1,0
XO_RO_SHI_RO_1024_S true 1,1,1,1,1,1,2,2,1,1
XO_RO_SHI_RO_1024_SS false 0,1,0,1,0,0,1,0,0,0
XO_RO_SHI_RO_1024_SS true 0,0,0,1,0,1,0,0,0,0
{noformat}
In this case {{XO_RO_SHI_RO_1024_S}} systematically fails the {{LinearComp}}
test when bit reversed.
h3. Upper 32-bits
{noformat}
RNG Bit-reversed TestU01 (BigCrush)
XO_RO_SHI_RO_1024_PP false 1,0,1,0,0,1,0,0,1,0
XO_RO_SHI_RO_1024_PP true 0,0,0,0,1,1,1,0,0,0
XO_RO_SHI_RO_1024_S false 1,0,0,0,0,0,0,1,0,0
XO_RO_SHI_RO_1024_S true 1,1,0,0,0,0,0,3,0,0
XO_RO_SHI_RO_1024_SS false 0,2,0,0,1,0,1,0,1,0
XO_RO_SHI_RO_1024_SS true 2,0,1,0,1,0,0,0,1,0
{noformat}
h3. 64-bits, alternating upper then lower
This is using the full sequence in the generated order as done by the default
implementation for any provider of 64-bit output.
{noformat}
RNG TestU01 (BigCrush)
XO_RO_SHI_RO_1024_PP 1,0,0,0,0,0,0,1,0,0
XO_RO_SHI_RO_1024_S 0,0,0,0,0,1,0,1,0,0
XO_RO_SHI_RO_1024_SS 2,0,0,0,0,0,1,0,1,0
{noformat}
Here failures are reduced compared to using only the upper or lower 32-bits.
h3. Conclusion
The {{XO_RO_SHI_RO_1024_S}} is recommended by the authors as a generator for
floating-point computation where the lower bits are discarded. These results
show a systematic failure of the lower 32-bits when bit reversed and agree with
the authors recommendation.
The {{_PP}} and {{_SS}} generators are recommended as all purpose generators.
These results show that either of those avoid systematic failures on BigCrush
with upper or lower bits. Given that the {{_PP}} generator is marginally faster
and also shows fewer spurious failures that the {{_SS}} variant I recommended
switching to {{XO_RO_SHI_RO_1024_PP}} for the SeedFactory.
> Change SeedFactory random source
> --------------------------------
>
> Key: RNG-122
> URL: https://issues.apache.org/jira/browse/RNG-122
> Project: Commons RNG
> Issue Type: Improvement
> Components: simple
> Affects Versions: 1.3
> Reporter: Alex Herbert
> Assignee: Alex Herbert
> Priority: Trivial
>
> The SeedFactory uses {{XOR_SHIFT_1024_S}}. This has a systematic failure in
> BigCrush using the lower 32-bit bit in reverse order:
> [Xorshift1024*, xorshift1024+, xorshift128+ and xoroshiro128+ fail
> statistical tests for
> linearity|https://www.sciencedirect.com/science/article/pii/S0377042718306265?dgcid=author]
> I have tested {{XOR_SHIFT_1024_S}} and {{XOR_SHIFT_1024_S_PHI}} using 10 runs
> of BigCrush and see the following results:
> {noformat}
> RNG Bit-reversed TestU01 (BigCrush)
> XOR_SHIFT_1024_S true 1,2,1,1,1,1,2,2,1,1
> XOR_SHIFT_1024_S_PHI true 1,1,1,1,2,1,1,1,1,1
> RandomSource,Bit-reversed,Test,Failures,Failed
> XOR_SHIFT_1024_S,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S,true,TestU01 (BigCrush),2,68 MatrixRank, L=1000, r=0|80
> LinearComp, r = 0
> XOR_SHIFT_1024_S,true,TestU01 (BigCrush),2,25 ClosePairs mNP2S, t = 16|80
> LinearComp, r = 0
> XOR_SHIFT_1024_S,true,TestU01 (BigCrush),2,74 RandomWalk1 M (L=50, r=0)|80
> LinearComp, r = 0
> XOR_SHIFT_1024_S_PHI,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S_PHI,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S_PHI,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S_PHI,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S_PHI,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S_PHI,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S_PHI,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S_PHI,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S_PHI,true,TestU01 (BigCrush),1,80 LinearComp, r = 0
> XOR_SHIFT_1024_S_PHI,true,TestU01 (BigCrush),2,80 LinearComp, r = 0|80
> LinearComp, r = 0
> {noformat}
> This shows a systematic failure of the {{LinearComp}} test as observed in the
> paper linked above.
> Update the SeedFactory to use a different generator.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
