[issue21470] Better seeding for the random module

Changes by Christian Heimes : -- nosy: +christian.heimes ___ Python tracker ___ ___

[issue21470] Better seeding for the random module

Larry Hastings added the comment: (If the past few weeks have taught us *anything*, it's that we can't look to OpenSSL to learn best practices.) -- nosy: +larry stage: - resolved ___ Python tracker rep...@bugs.python.org

[issue21470] Better seeding for the random module

Charles-François Natali added the comment: Thanks for the explanations! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21470 ___ ___

[issue21470] Better seeding for the random module

Marc-Andre Lemburg added the comment: neologix: According to man rand(3ssl), OpenSSL uses an internal state of 1023 bytes for the RNG. You only see it reading 32 bytes from /dev/urandom in the strace because it has already loaded 1024 bytes from the RNG state file ~/.rng before adding

[issue21470] Better seeding for the random module

Antoine Pitrou added the comment: Is ~/.rnd any kind of serious? It hasn't been modified since two weeks on my system (which is rebooted every day). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21470

[issue21470] Better seeding for the random module

Charles-François Natali added the comment: According to man rand(3ssl), OpenSSL uses an internal state of 1023 bytes for the RNG. You only see it reading 32 bytes from /dev/urandom in the strace because it has already loaded 1024 bytes from the RNG state file ~/.rng before adding

[issue21470] Better seeding for the random module

Marc-Andre Lemburg added the comment: On 13.05.2014 11:06, Antoine Pitrou wrote: Is ~/.rnd any kind of serious? It hasn't been modified since two weeks on my system (which is rebooted every day). The file is apparently only updated if you use one the OpenSSL commands which needs random

[issue21470] Better seeding for the random module

Tim Peters added the comment: Crytpo generators are a whole different world, and I wouldn't listen to anyone save a bona fide expert in that field. Plausible: the hardest thing OpenSSL has to do is generate secure RSA keys. But the bit length of an RSA key can't be taken at face value:

[issue21470] Better seeding for the random module

Roundup Robot added the comment: New changeset 7b5265752942 by Raymond Hettinger in branch '2.7': Issue #21470: Do a better job seeding the random number generator http://hg.python.org/cpython/rev/7b5265752942 -- nosy: +python-dev ___ Python tracker

[issue21470] Better seeding for the random module

Roundup Robot added the comment: New changeset c203df907092 by Raymond Hettinger in branch '3.4': Issue #21470: Do a better job seeding the random number generator http://hg.python.org/cpython/rev/c203df907092 -- ___ Python tracker

[issue21470] Better seeding for the random module

Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21470 ___

[issue21470] Better seeding for the random module

STINNER Victor added the comment: MT is equidistributed. This a major point in its favor but also implies that there are long stretches of uninteresting sequences. When we seed with only a subset the state space, there is a risk of systematically landing in those stretches. What is an

[issue21470] Better seeding for the random module

Tim Peters added the comment: [haypo] What is an uninteresting sequence? What are the problem of these sequences? A sequence that would greatly surprise a user. For example, if you generate 32-bit ints from the Twister in one obvious way, there are starting places where you'll get 623

[issue21470] Better seeding for the random module

STINNER Victor added the comment: A sequence that would greatly surprise a user. No user complained past years. I don't think that we should worry so much, because it looks like reading more data from /dev/urandom can be a more serious and concrete issue. --

[issue21470] Better seeding for the random module

Tim Peters added the comment: [haypo] No user complained past years. Raymond said We've previously had this problem with MT (since resolved, where it is was landed in a very non-random zone). Do you believe he was wrong? I don't think that we should worry so much, because it looks like

[issue21470] Better seeding for the random module

Antoine Pitrou added the comment: There is no useful theory that allows us to predict the characteristics of the produced sequences from a set of possible seeds, so limiting the set of possible seeds is potentially dangerous. I still find it difficult to understand where is the said danger.

[issue21470] Better seeding for the random module

Tim Peters added the comment: [pitrou] I still find it difficult to understand where is the said danger. The theoretical properties that make the Twister so attractive were all proved based on mathematical analysis of its entire period. The only way to get at the whole period is to allow

[issue21470] Better seeding for the random module

Antoine Pitrou added the comment: The theoretical properties that make the Twister so attractive were all proved based on mathematical analysis of its entire period. The only way to get at the whole period is to allow for all possible seeds. If the seeds Python can use are drawn from a

[issue21470] Better seeding for the random module

Tim Peters added the comment: Thanks for the explanation. It's much clearer now. Maybe, but it's also overblown - LOL ;-) That is, no matter what the starting seed, the user will see a microscopically tiny span of the Twister's entire period. So all those provably correct properties that

[issue21470] Better seeding for the random module

Charles-François Natali added the comment: Tim, any idea why openssl, openssh Co get away with just 32 bytes of seed read from /dev/urandom? Is it because of a much smaller state space of the underlying CSPRNG? -- ___ Python tracker

[issue21470] Better seeding for the random module

Charles-François Natali added the comment: The default seeding for the random module currently used 32 bytes from urandom() to create the initial state of the random number generator. This is far less than the number of possible states 2**19937-1. I'm not a cryptography expert, but IMO 32

[issue21470] Better seeding for the random module

Raymond Hettinger added the comment: Several thoughts: * We're not reading urandom a huge number of times per second. This is just one read of 2,500 bytes. What Ted is talking about and what we're doing are as different as night and day. * We're also not doing this in a loop. It is just

[issue21470] Better seeding for the random module

Antoine Pitrou added the comment: http://www.2uo.de/myths-about-urandom/ Thanks, interesting read. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21470 ___

[issue21470] Better seeding for the random module

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21470 ___

[issue21470] Better seeding for the random module

Mark Dickinson added the comment: Raymond: Functions as simple as shuffle() eat through the possibilities very quickly. Can you elaborate on this? Are there example scenarios where seeding with 32 bytes isn't likely to be enough? In the case of shuffle, for a large list, if you do a seed

[issue21470] Better seeding for the random module

Charles-François Natali added the comment: * We're not reading urandom a huge number of times per second. This is just one read of 2,500 bytes. What Ted is talking about and what we're doing are as different as night and day. * We're also not doing this in a loop. It is just once when

[issue21470] Better seeding for the random module

Raymond Hettinger added the comment: 32 bytes seeding is more than enough. Not enough to cover *our* state space (2 ** 19937 - 1). This tracker item boils down to balancing fear that something bad will happen when you call urandom(2500) versus having seeding sufficient to cover the state

[issue21470] Better seeding for the random module

Raymond Hettinger added the comment: http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/VERSIONS/C-LANG/MersenneTwister.h http://www.omnetpp.org/doc/omnetpp/api/mersennetwister_8h_source.html -- ___ Python tracker rep...@bugs.python.org

[issue21470] Better seeding for the random module

Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- Removed message: http://bugs.python.org/msg218288 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21470 ___

[issue21470] Better seeding for the random module

Raymond Hettinger added the comment: http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/VERSIONS/C-LANG/MersenneTwister.h http://www.omnetpp.org/doc/omnetpp/api/mersennetwister_8h_source.html http://www0.cs.ucl.ac.uk/staff/d.jones/GoodPracticeRNG.pdf --

[issue21470] Better seeding for the random module

STINNER Victor added the comment: The default seeding for the random module currently used 32 bytes from urandom() to create the initial state of the random number generator. This is far less than the number of possible states 2**19937-1. I suggest to document how the PRNG is initialized

[issue21470] Better seeding for the random module

Antoine Pitrou added the comment: What constitutes enough is a value judgment that many vary from application to application. For some applications, a much weaker PRNG would suffice, but we decided long ago that we wanted the full power of MT. I don't really understand for which

[issue21470] Better seeding for the random module

Raymond Hettinger added the comment: Looking back over this tracker item, I realize that I didn't elaborate sufficiently on the problem being addressed: MT is equidistributed. This a major point in its favor but also implies that there are long stretches of uninteresting sequences. When we

[issue21470] Better seeding for the random module

Tim Peters added the comment: [neologix] some code spawns many processes per second (see recent discussion on python-dev). But that doesn't imply they're seeding the random module many times per second, right? Seeding isn't part of Python initialization, it's part of importing the `random`

[issue21470] Better seeding for the random module

Antoine Pitrou added the comment: [neologix] some code spawns many processes per second (see recent discussion on python-dev). But that doesn't imply they're seeding the random module many times per second, right? Seeding isn't part of Python initialization, it's part of importing the

[issue21470] Better seeding for the random module

New submission from Raymond Hettinger: The default seeding for the random module currently used 32 bytes from urandom() to create the initial state of the random number generator. This is far less than the number of possible states 2**19937-1. Changing the default seed to a larger number

[issue21470] Better seeding for the random module

Changes by Alex Gaynor alex.gay...@gmail.com: -- nosy: +alex ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21470 ___ ___ Python-bugs-list mailing

[issue21470] Better seeding for the random module

Tim Peters added the comment: +1, although it could really use a comment explaining that 2500 bytes was chosen to be = the Twister's 19937 internal bits of state. Otherwise it looks as arbitrary as 32 did ;-) -- nosy: +tim.peters ___ Python

[issue21470] Better seeding for the random module

Antoine Pitrou added the comment: I'm not sure it is good practice to read that many bytes from /dev/urandom. Quoting the Linux man page for /dev/urandom: The kernel random-number generator is designed to produce a small amount of high-quality seed material to seed a

[issue21470] Better seeding for the random module

Changes by Ezio Melotti ezio.melo...@gmail.com: -- nosy: +ezio.melotti ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21470 ___ ___

[issue21470] Better seeding for the random module

Antoine Pitrou added the comment: Also, the repletion rate of the entropy pool seems quite slow actually (around 10 bytes per second?). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21470

[issue21470] Better seeding for the random module

Donald Stufft added the comment: Depleting the entropy pool is sort of a nonsense idea that /dev/random has. Nobody should ever be worried about it and nobody should ever use /dev/random. The manpage is wrong and has continued to be wrong because of historical reasons and the people involved

[issue21470] Better seeding for the random module

Raymond Hettinger added the comment: http://www.2uo.de/myths-about-urandom/ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21470 ___ ___

[issue21470] Better seeding for the random module

Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- versions: +Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21470 ___ ___