Re: class SplittableRandom

2013-08-09 Thread Doug Lea
On 08/09/13 11:43, Aleksey Shipilev wrote: On 08/09/2013 04:29 PM, Doug Lea wrote: Currently at: http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/main/java/util/SplittableRandom.java?view=log Looks good. Stupid questions below: * Why use InetAddress.getLocalHost() for the initial seed

Re: class SplittableRandom

2013-08-09 Thread Aleksey Shipilev
On 08/09/2013 04:29 PM, Doug Lea wrote: > Currently at: > http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/main/java/util/SplittableRandom.java?view=log Looks good. Stupid questions below: * Why use InetAddress.getLocalHost() for the initial seed? I understand you don't want to use Secure

Re: class SplittableRandom

2013-08-09 Thread Doug Lea
After exploring a bunch of algorithmic variants, we committed to one of the simplest for consideration for inclusion in JDK8. Thanks again to everyone sending comments and suggestions that led us to investigate alternatives. The API is unchanged. While we cannot exhaustively quality-check all p

Re: class SplittableRandom

2013-07-30 Thread Doug Lea
On 07/29/13 14:29, Peter Levart wrote: On 07/16/2013 06:01 PM, Guy Steele wrote: So I think we will not hit any really bad gamma values in practice. Would it be possible to skip gammas with less than 12 or more than 52 bits? Or better, eliminate dependence on bit density. I'm working on

Re: class SplittableRandom

2013-07-30 Thread Peter Levart
On 07/30/2013 04:25 PM, Paul Sandoz wrote: Hi Peter, Given that the user cannot specify gamma values and the for the first 2^16+1 gamma values the minimum bit count is 15, the maximum bit count is 50 and the minimum width is 45, is that not sufficient? i.e. is it likely there will be more t

Re: class SplittableRandom

2013-07-30 Thread Paul Sandoz
Hi Peter, Given that the user cannot specify gamma values and the for the first 2^16+1 gamma values the minimum bit count is 15, the maximum bit count is 50 and the minimum width is 45, is that not sufficient? i.e. is it likely there will be more than 65537 splits? In the streams API one wou

Re: class SplittableRandom

2013-07-29 Thread Peter Levart
On 07/16/2013 06:01 PM, Guy Steele wrote: On Jul 16, 2013, at 3:07 AM, Martin Buchholz wrote: Read Appleby and Stafford ... Hmmm mix32 has almost the same job as mix64 - have all the bits in the seed affect all the bits in the output, so the obvious implementation is mix32() { return (i

Re: class SplittableRandom

2013-07-21 Thread Doug Lea
(Catching up...) On 07/19/13 14:00, Mike Duigou wrote: Using a shared static SecureRandom.generateSeed() to generate the seed is probably the easiest way to get a good seed that is portable. I agree and updated accordingly. I have previous experience mucking in this area. Attempting to use

Re: class SplittableRandom

2013-07-19 Thread David Chase
On 2013-07-19, at 2:00 PM, Mike Duigou wrote: > Attempting to use SecureRandom will likely encounter order of initialization > issues as it is loaded from security providers and is only > functional/available very late in the VM boot process. Solutions which use > lousy seeding until SecureRan

Re: class SplittableRandom

2013-07-19 Thread Mike Duigou
On Jul 19 2013, at 10:43 , Kasper Nielsen wrote: > On Fri, Jul 19, 2013 at 5:10 PM, Guy Steele wrote: > >> >> On Jul 19, 2013, at 1:05 AM, Kasper Nielsen wrote: >> >> Thanks so much for your feedback! Your points are well taken. >> >> I agree that this is a problem, and in fact have alread

Re: class SplittableRandom

2013-07-19 Thread Kasper Nielsen
On Fri, Jul 19, 2013 at 5:10 PM, Guy Steele wrote: > > On Jul 19, 2013, at 1:05 AM, Kasper Nielsen wrote: > > Thanks so much for your feedback! Your points are well taken. > > I agree that this is a problem, and in fact have already looked at ways to > get > other environmental information to m

Re: class SplittableRandom

2013-07-19 Thread Guy Steele
On Jul 19, 2013, at 1:05 AM, Kasper Nielsen wrote: Thanks so much for your feedback! Your points are well taken. > Telling people that your random source passes the die-hard test but at the > same time only using the current time as the seed is just pure evil. > Imagine spinning up a couple of

Re: class SplittableRandom

2013-07-18 Thread Kasper Nielsen
Telling people that your random source passes the die-hard test but at the same time only using the current time as the seed is just pure evil. Imagine spinning up a couple of thousands images with a JVM on Amazon. You are going to see collisions among the seed values a lot sooner than you think. E

Re: RFR 8020292 :Re: class SplittableRandom

2013-07-17 Thread Mike Duigou
Hello all; Paul is on vacation (well deserved) and Doug/Guy/others have been continuing to work on SplittableRandom. As the code is continuing to evolve quickly I won't immediately post an update to this list but encourage those interested to follow along with updates either in the lambda repo:

Re: class SplittableRandom

2013-07-16 Thread Doug Lea
On 07/16/13 12:55, Mike Duigou wrote: I worry that it will be difficult/impractical for those who desire a different algorithm to replace SplittableRandom with their own (P)RNG whether it be for secure randomness or better statistical properties. Since SplittableRandom doesn't implement any inte

Re: class SplittableRandom

2013-07-16 Thread Guy Steele
On Jul 16, 2013, at 2:57 PM, Guy Steele wrote: > . . . > So here's the punchline: this change makes my Monte Carlo calculation of pi > run over 19% faster! > Hard to believe. It's certainly avoiding most of the hard work in > addGammaModGeorge, > and perhaps also making branch prediction total

Re: class SplittableRandom

2013-07-16 Thread Doug Lea
On 07/16/13 14:57, Guy Steele wrote: So instead I took the latest version and modified the generation of gamma values to use a 56-bit GAMMA_GAMMA value and perform arithmetic modulo 2^56-5 ... So here's the punchline: this change makes my Monte Carlo calculation of pi run over 19% faster! Ha

Re: class SplittableRandom

2013-07-16 Thread Guy Steele
On Jul 15, 2013, at 4:13 PM, Martin Buchholz wrote: > > On Mon, Jul 15, 2013 at 9:59 AM, Martin Buchholz wrote: >> >> Also, if gamma is much less than 2^64 (say 2^56; that number of gammas >> "should be enough for anybody"), then the above will be a little more >> efficient since the wraparound

Re: class SplittableRandom

2013-07-16 Thread Doug Lea
On 07/16/13 12:41, Mike Duigou wrote: On Jul 16 2013, at 05:37 , Doug Lea wrote: Mike Duigou: It's probably reasonable to let lambda and webrev versions lag a few days until a bit more stable. Until then, see http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/main/java/util/SplittableRa

Re: class SplittableRandom

2013-07-16 Thread Guy Steele
Maybe "SplittableRandom" should be an interface, and the current implementation called, say, "SplittableRandom64" (for its 64-bit seed). --Guy On Jul 16, 2013, at 12:55 PM, Mike Duigou wrote: > > On Jul 15 2013, at 16:11 , Doug Lea wrote: > >> On 07/15/13 12:59, Martin Buchholz wrote: >>> If

Re: class SplittableRandom

2013-07-16 Thread Mike Duigou
On Jul 15 2013, at 16:11 , Doug Lea wrote: > On 07/15/13 12:59, Martin Buchholz wrote: >> If we make the wraparound point for seeds Long.MIN_VALUE instead of 0L, then >> we >> can optimize addGammaModGeorge. > > This was a great suggestion; thanks! > >> >> I continue to worry that we are prod

Re: class SplittableRandom

2013-07-16 Thread Mike Duigou
On Jul 16 2013, at 05:37 , Doug Lea wrote: > > I incorporated updates stemming from yesterday's SplittableRandom > list and off-list traffic. Thanks as always for all the helpfulness. > > Mike Duigou: It's probably reasonable to let lambda and webrev > versions lag a few days until a bit more s

Re: class SplittableRandom

2013-07-16 Thread Guy Steele
On Jul 16, 2013, at 3:07 AM, Martin Buchholz wrote: > > Read Appleby and Stafford ... > > Hmmm mix32 has almost the same job as mix64 - have all the bits in the > seed affect all the bits in the output, so the obvious implementation is > > mix32() { return (int) mix64(); } > or more cons

Re: class SplittableRandom

2013-07-16 Thread Doug Lea
I incorporated updates stemming from yesterday's SplittableRandom list and off-list traffic. Thanks as always for all the helpfulness. Mike Duigou: It's probably reasonable to let lambda and webrev versions lag a few days until a bit more stable. Until then, see http://gee.cs.oswego.edu/cgi-bin

Re: RFR 8020292 :Re: class SplittableRandom

2013-07-15 Thread Doug Lea
On 07/15/13 09:02, Aleksey Shipilev wrote: OK, early performance data follows. ... - 64 bit: on moderately saturated machine (1, 16 threads), SR double is marginally faster than TLR, and SR int/long is significantly faster than TLR - 32 bit: TLR is vastly faster in int/long standalone scenar

Re: class SplittableRandom

2013-07-15 Thread Doug Lea
On 07/15/13 19:06, Martin Buchholz wrote: Another experiment that provides bit-full seed but poor gamma Keeping in mind that we won't support the seed+gamma constructor, For what value of K would you be content to know that the generators associated with the first K splits for any SplittableRan

Re: class SplittableRandom

2013-07-15 Thread Doug Lea
On 07/15/13 12:59, Martin Buchholz wrote: If we make the wraparound point for seeds Long.MIN_VALUE instead of 0L, then we can optimize addGammaModGeorge. This was a great suggestion; thanks! I continue to worry that we are producing a PRNG that will end up not being acceptable to the Monte C

Re: class SplittableRandom

2013-07-15 Thread Doug Lea
On 07/15/13 16:13, Martin Buchholz wrote: I'll be your loyal adversary today, trying to break SplittableRandom. Let's make it easier by providing a low-level constructor: /** Testing */ public SplittableRandom(long seed, long gamma, Error ) { Now you see why we don't do tha

Re: class SplittableRandom

2013-07-15 Thread Guy Steele
On Jul 15, 2013, at 4:13 PM, Martin Buchholz wrote: > I'll be your loyal adversary today, trying to break SplittableRandom. Thanks so much! This is exactly the sort of skepticism we need. > Let's make it easier by providing a low-level constructor: > >/** Testing */ >public Spli

Re: RFR 8020292 :Re: class SplittableRandom

2013-07-15 Thread David Chase
On 2013-07-15, at 9:02 AM, Aleksey Shipilev wrote: > The conclusion is: SR is a reasonable alternative with no obvious > performance show-stoppers. That said, TLR is also reasonably fast at > this point, and further improvements to TLR will probably beat the SR or > tie it on most scenarios. I t

Re: class SplittableRandom

2013-07-15 Thread Doug Lea
On 07/15/13 00:24, Martin Buchholz wrote: Thoughts: I was sad when ThreadLocalRandom reused the lousy pseudorandom implementation from Random. I continue to think we can do better. Yes, I'm working on it. It would be nice if we could guarantee a minimum period of 2^64. OK. I cannot think

Re: RFR 8020292 :Re: class SplittableRandom

2013-07-15 Thread Aleksey Shipilev
On 07/15/2013 05:43 PM, Peter Levart wrote: > On 07/15/2013 03:02 PM, Aleksey Shipilev wrote: >> The results are here [2]. Notable results follow. > Somehow the files in the directories under: > > http://cr.openjdk.java.net/~shade/8020292/data/ > > ... are not accessible (403 - Forbidden), althou

Re: RFR 8020292 :Re: class SplittableRandom

2013-07-15 Thread Peter Levart
On 07/15/2013 03:02 PM, Aleksey Shipilev wrote: The results are here [2]. Notable results follow. Hi Aleksey, Somehow the files in the directories under: http://cr.openjdk.java.net/~shade/8020292/data/ ... are not accessible (403 - Forbidden), although the sub-directories are browsable...

Re: RFR 8020292 :Re: class SplittableRandom

2013-07-15 Thread Aleksey Shipilev
On 07/12/2013 09:26 PM, Paul Sandoz wrote: > Here is the webrev: > http://cr.openjdk.java.net/~psandoz/tl/JDK-8020292-splittable-random/webrev/ OK, early performance data follows. I've used a simple benchmark [1] to estimate the standalone costs for getting int/long/double, as well as the usage

Re: class SplittableRandom

2013-07-14 Thread Doug Lea
(Sorry to postpone answering this one while tied up in logistics...) On 07/11/13 08:23, Aleksey Shipilev wrote: On 07/11/2013 04:16 PM, Doug Lea wrote: (I've been contemplating re-exploring alternatives in TLR, but the options are more limited because it is a subclass of Random. Which made sens

Re: class SplittableRandom

2013-07-12 Thread Doug Lea
On 07/12/13 14:29, Mike Duigou wrote: Most specifically, do we want to commit to a seeded version promising a deterministic sequence that would preclude future changes Thanks, this a good spec clarification question, about the scope of: * Creates a new SplittableRandom instance using the

Re: class SplittableRandom

2013-07-12 Thread Mike Duigou
On Jul 12 2013, at 10:07 , Doug Lea wrote: > [Also posted on concurrency-interest.] > > Thanks to all those sending quick feedback, both on and off lists! > We're increasingly confident about the value of adding > SplittableRandom, so will proceed with integration request. I'm sure > that we'll

Re: RFR 8020292 :Re: class SplittableRandom

2013-07-12 Thread Paul Sandoz
On Jul 12, 2013, at 7:26 PM, Paul Sandoz wrote: > Hi, > > Thanks Doug. > > Here is the webrev: > > http://cr.openjdk.java.net/~psandoz/tl/JDK-8020292-splittable-random/webrev/ > In my haste i forgot to mention this patch is based off: http://cr.openjdk.java.net/~psandoz/tl/JDK-8019395-S

RFR 8020292 :Re: class SplittableRandom

2013-07-12 Thread Paul Sandoz
Hi, Thanks Doug. Here is the webrev: http://cr.openjdk.java.net/~psandoz/tl/JDK-8020292-splittable-random/webrev/ Paul. On Jul 12, 2013, at 7:07 PM, Doug Lea wrote: > [Also posted on concurrency-interest.] > > Thanks to all those sending quick feedback, both on and off lists! > We're incr

Re: class SplittableRandom

2013-07-12 Thread Doug Lea
[Also posted on concurrency-interest.] Thanks to all those sending quick feedback, both on and off lists! We're increasingly confident about the value of adding SplittableRandom, so will proceed with integration request. I'm sure that we'll introduce incremental improvements during OpenJDK8 bake-

Re: class SplittableRandom

2013-07-11 Thread Doug Lea
On 07/11/13 15:50, Jeff Hain wrote: One could use that instead, which doesn't have that problem, and provides values in [0,1-1/2^53], with 1/2^53 granularity: public double nextDouble() { return (this.nextLong() & ((1L<<53)-1)) * (1.0/(1L<<53)); } Thanks very much! We change

Re: class SplittableRandom

2013-07-11 Thread Jeff Hain
Doug Lea wrote: >>    new Subtask(new MySequentialRandom(currentTaskRandom.nextLong())).fork() >One way to think of SplittableRandom is that it makes your idea >behind this code actually work. As it stands, if you were to do >this with java.util.Random, both would generate the same sequence. What

Re: class SplittableRandom

2013-07-11 Thread Doug Lea
On 07/11/13 15:50, Jeff Hain wrote: Doug Lea wrote: >Q: Why isn't SplittableRandom a subclass of Random? >A: Class Random requires thread-safety in its spec. It would be >nonsensical for SplittableRandom to comply. Random Javadoc states "Instances of java.util.Random are threadsafe." but Thre

Re: class SplittableRandom

2013-07-11 Thread Jeff Hain
Hello. 1) extending Random Doug Lea wrote: >Q: Why isn't SplittableRandom a subclass of Random? >A: Class Random requires thread-safety in its spec. It would be >nonsensical for SplittableRandom to comply. Random Javadoc states "Instances of java.util.Random are threadsafe." but ThreadLocalRan

Re: class SplittableRandom

2013-07-11 Thread Aleksey Shipilev
(I've been distracted by other high priority stuff, and only starting to explore SR, sorry for dumb questions below) On 07/11/2013 04:16 PM, Doug Lea wrote: > (I've been contemplating re-exploring alternatives in TLR, > but the options are more limited because it is a subclass > of Random. Which m

Re: class SplittableRandom

2013-07-11 Thread Doug Lea
On 07/11/13 03:36, Peter Levart wrote: Hi Doug, I confess I'm not an expert in PRNGs, but I'll try to ask a hopefully no-nonsense question anyway. SplittableRandom seems to be a kind of PR-PRNG-G - a PseudoRandom-PseudoRandomGenerator-Generator. In a sense that it's split() method returns an ins

Re: class SplittableRandom

2013-07-11 Thread Peter Levart
Hi Doug, I confess I'm not an expert in PRNGs, but I'll try to ask a hopefully no-nonsense question anyway. SplittableRandom seems to be a kind of PR-PRNG-G - a PseudoRandom-PseudoRandomGenerator-Generator. In a sense that it's split() method returns an instance that represents a PRNG which p

Re: class SplittableRandom

2013-07-10 Thread Doug Lea
On 07/10/13 16:49, Aleksey Shipilev wrote: (juggling an Occam's razor) Thanks keep it sharp; the more skepticism the better. D'oh, of course it is ludicrously faster than j.u.Random. I am not convinced it beats TLR though, given TLR is super-optimized now in JDK 8. Are you really implying yo

Re: class SplittableRandom

2013-07-10 Thread Aleksey Shipilev
(juggling an Occam's razor) Hi Doug, On 10.07.2013, at 23:13, Doug Lea wrote: > This post serves as a request for comment, with shorter than usual > turnaround (a couple of days) before considering a request to > integrate into OpenJDK 8. So, please take a look. I've glanced over the paper this