Kim Jin Chul has posted comments on this change. ( http://gerrit.cloudera.org:8080/8355 )
Change subject: IMPALA-5754: Improve randomness of rand()/random() ...................................................................... Patch Set 14: (1 comment) Thanks a lot Jim for your guidance to solve this issue. http://gerrit.cloudera.org:8080/#/c/8355/14/be/src/exprs/math-functions-ir.cc File be/src/exprs/math-functions-ir.cc: http://gerrit.cloudera.org:8080/#/c/8355/14/be/src/exprs/math-functions-ir.cc@165 PS14, Line 165: mt19937* generator = ctx->Allocate<mt19937>(); > It might take a while. It initializes a state of 624 32-bit values, as you I applied PCG-random on the recent patch set. The high memory consumption issue should disappear when we adopts pcg-32bit: - Current: 8 bytes - PCG-32bit: 16 bytes - Mersenne Twister 32bit(a.k.a. mt19937): 5,000 bytes Regarding the perspective of performance of the generation, 1. Here is a comparison between pcg32 and pcg32-fast. pcg32-fast is faster than pcg32, but pcg32-fast shows a slightly less statistically good output than pcg32. (http://www.pcg-random.org/using-pcg-cpp.html#bit-generators-with-64-bit-state) 2. Here is a comparison between pcg32 and Mersenne Twister 32bit. pcg32(method: PCG XSH RS 64/32 (MCG)) is about 3.5x times faster than Mersenne Twister 32bit. (http://www.pcg-random.org/pdf/hmc-cs-2014-0905.pdf) My conclusion is pcg32 is promising because it shows good statistical quality, good performance and very compact space usage. -- To view, visit http://gerrit.cloudera.org:8080/8355 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idafdd5fe7502ff242c76a91a815c565146108684 Gerrit-Change-Number: 8355 Gerrit-PatchSet: 14 Gerrit-Owner: Kim Jin Chul <[email protected]> Gerrit-Reviewer: Attila Jeges <[email protected]> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple <[email protected]> Gerrit-Reviewer: Kim Jin Chul <[email protected]> Gerrit-Reviewer: Michael Ho <[email protected]> Gerrit-Comment-Date: Thu, 07 Dec 2017 00:32:57 +0000 Gerrit-HasComments: Yes
