[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

STINNER Victor added the comment: > Re-reviewing your patch, I agree the What's New comment is still accurate. Thanks for double checking ;-) -- ___ Python tracker

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Nick Coghlan added the comment: Sorry, with all the different proposals kicking around, I somehow got the impression we'd reverted entirely to just reading from /dev/urandom without ever using the new syscall. Re-reviewing your patch, I agree the What's New comment is still accurate.

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

STINNER Victor added the comment: Nick Coghlan: "One last fix needed to fully revert this is to remove the mention from the Python 3.5 What's New documentation: https://docs.python.org/3.5/whatsnew/3.5.html#os; This sentence? "The urandom() function now uses the getrandom() syscall on Linux

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Nick Coghlan added the comment: One last fix needed to fully revert this is to remove the mention from the Python 3.5 What's New documentation: https://docs.python.org/3.5/whatsnew/3.5.html#os -- ___ Python tracker

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Larry Hastings added the comment: Colm Buckley: I've read the code, *and* stepped through it, and AFAICT it is no longer even possible for Python on Linux to call getrandom() in a blocking way. Thanks for doing this! I'm marking the issue as closed. -- stage: patch review ->

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Larry Hastings added the comment: I just posted to python-dev and asked Guido to make a BDFL ruling. I only represented my side, both because I worried I'd do a bad job of representing *cough* literally everybody else *cough*, and because it already took me so long to write the email. All

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Larry Hastings added the comment: > Regardless of the behavior of os.urandom (and 'import random'), is it agreed > that the current state of _PyRandom_Init is acceptable for 3.5.2? I'll get back to you with a specific yes or no. What I want is that it the behavior removed where "import

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Changes by Nick Coghlan : -- nosy: +ncoghlan ___ Python tracker ___ ___ Python-bugs-list

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

STINNER Victor added the comment: > The current behavior (as of 9de508dc4837) is that it will never block on > Linux, but could still block on other OS if called before /dev/urandom is > initialized. In practice, only Linux is impacted. See the rationale:

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

STINNER Victor added the comment: I opened the issue #27272: "random.Random should not read 2500 bytes from urandom". -- ___ Python tracker ___

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: Larry - Regardless of the behavior of os.urandom (and 'import random'), is it agreed that the current state of _PyRandom_Init is acceptable for 3.5.2? The current behavior (as of 9de508dc4837) is that it will never block on Linux, but could still block on

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Changes by ppperry : -- title: Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing /arguinthe random module on getrandom() -> Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing /arguinthe random module on getrandom()

Marc-Andre Lemburg added the comment: On 08.06.2016 22:49, Larry Hastings wrote: > > Third, because the os module is in general a thin wrapper over what the OS > provides, I disapprove of "cryptorandom()" and "pseudorandom()" going into > the os module. There are no functions with these

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing /arguinthe random module on getrandom()

Larry Hastings added the comment: I am increasingly convinced that I'm right. -- First, consider that the functions in the os module, as a rule, are a thin shell over the equivalent function provided by the operating system. If Python exposes a function called os.XYZ(), and it calls the OS,

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing /arguinthe random module on getrandom()

Colm Buckley added the comment: @pitti - We already discussed this; there are cases where it's not practical to set an environment variable. The discussion eventually converged on "it is not desirable that Python should block on startup, regardless of system RNG status". Re: the triggering

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing /arguinthe random module on getrandom()

Martin Pitt added the comment: > you could give some kind of command-line flag That already exists -- set PYTHONHASHSEED=0. > But I'll let someone else have the joys of negotiating with Lennart, and I > won't blame the Python devs if using GRND_NONBLOCK unconditionally is less > painful than

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

STINNER Victor added the comment: I spent almost my whole day to read this issue, some related issues, and some more related links. WOW! Amazing discussing. Sorry that Christian decided to quit the discussion (twice) :-( Here is my summary: http://haypo-notes.readthedocs.io/pep_random.html

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: Just to re-state; I think we have three problems: 1) _Py_HashSecret initialization blocking. Affects all Python invocations; already a substantial issue on Debian testing track (90s startup delay). * there seems to be general agreement that this does not need a

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

STINNER Victor added the comment: >> The current behavior is that Python *will not start at all* if getrandom() >> blocks (because the hash secret initialization fails). > It starts jsut fine, it just can possible takes awhile. In my experience, connecting to a VM using SSH with low entropy

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: Victor - Yes, it is possible for it to block forever - see the test I proposed for Ted upthread. The triggering case (systemd-crontab-generator) delays for 90 seconds, but is *killed* by systemd after that time; it doesn't itself time out. Colm --

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Theodore Tso added the comment: Oh --- and about people wondering whether os.random is being used for cryptographic purposes or not "most of the time" or not --- again, welcome to my world. I get complaints all the time from people who try to do "dd if=/dev/urandom of=/dev/hdX bs=4k" and

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Theodore Tso added the comment: One of the reasons why trying to deal with randomness is hard is because a lot of it is about trust. Did Intel backdoor RDRAND to help out the NSA? You might have one answer if you work for the NSA, and perhaps if you are willing to assume the worst about

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

STINNER Victor added the comment: Cory Benfield (msg267637): "if the purpose of this patch was to prevent long startup delays, *it failed*. On all the systems above os.urandom may continue to block system startup." I don't pretend fixing the issue on all operating systems. As stated in the

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

STINNER Victor added the comment: Donald: "Cory wasn't speaking about (non)blocking in general, but the case where (apparently) it's desired to not block even if that means you don't get cryptographically secure random in the CPython interpreter start up. (...)" Oh sorry, I misunderstood his

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Christian Heimes added the comment: I'm unsubscribing from this ticket for the second time. This form of discussion is becoming toxic for me because I strongly beliefe that it is not the correct way to handle a security-related problem. -- ___

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Changes by Christian Heimes : -- nosy: -christian.heimes ___ Python tracker ___ ___

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Marc-Andre Lemburg added the comment: Even though it may sound like a minor problem that os.urandom() blocks on startup, I think the problem is getting more and more something to consider given how systems are used nowadays. Today, we no longer have the case where you keep a system up and

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Donald Stufft added the comment: Larry, I would greatly prefer it if we would allow os.urandom to block on Linux, at least by default. This will make os.urandom behavior similarly on most modern platforms. The cases where this is going to matter are extreme edge cases, for most users they'll

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Christian Heimes added the comment: On 2016-06-08 11:39, STINNER Victor wrote: > > STINNER Victor added the comment: > > no-urandom-by-default.diff uses a very weak source of entropy for > random.Random :-( I'm fighting against weak sources of entropy since many > years... It is totally

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

STINNER Victor added the comment: Larry Hastings: > > Hi. I created the issue #27266 "Add block keyword-only optional parameter > > to os.urandom()" (...) > > Happy? > Probably not. (...) I replied on the issue #27266. Sorry I'm unable to follow this issue, there are too many messages now

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

STINNER Victor added the comment: no-urandom-by-default.diff uses a very weak source of entropy for random.Random :-( I'm fighting against weak sources of entropy since many years... This change introduces the bug similar to OpenSSL RAND_bytes() bug (two processes with the same pid can

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Larry Hastings added the comment: > Hi. I created the issue #27266 "Add block keyword-only optional parameter > to os.urandom()" which is compromise between all proposed solutions and > should fix *all* urandom issues ;-) > > * os.urandom() remains secure by default, as asked by our security

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Cory Benfield added the comment: > You're right, it's remotely possible that on platforms where /dev/urandom > could block, Python startup could therefore also block. And I'm not > proposing we fix that, as so far nobody has reported it as a problem. > > This suggests to me that yes I'm

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Larry Hastings added the comment: You're right, it's remotely possible that on platforms where /dev/urandom could block, Python startup could therefore also block. And I'm not proposing we fix that, as so far nobody has reported it as a problem. This suggests to me that yes I'm talking

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

STINNER Victor added the comment: Hi. I created the issue #27266 "Add block keyword-only optional parameter to os.urandom()" which is compromise between all proposed solutions and should fix *all* urandom issues ;-) * os.urandom() remains secure by default, as asked by our security experts *

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Cory Benfield added the comment: > What I'm trying to avoid here is the surprising situation where someone is > using Python on a system where /dev/urandom will never block, and > os.urandom() blocks. At this point I literally do not understand what issue we're trying to solve then. If the

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Larry Hastings added the comment: I don't know if anyone literally still uses BSD. But on FreeBSD, /dev/urandom can block. So let me revise my statement slightly. Developers on platform X know how *their* /dev/urandom behaves. They should rightly expect that os.urandom() is a thin wrapper

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: [[ Larry - thanks for the Mercurial pointers. I was starting from the Debian sources because I initially assumed this was a Debian problem. Will switch to hg in future. ]] -- ___ Python tracker

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Christian Heimes added the comment: Larry, /dev/urandom blocks on BSD when it hasn't been seeded yet. But it looks like we can use sysctl to fetch the seed state from kern.random.sys.seeded. https://www.freebsd.org/cgi/man.cgi?query=random=4 The software generator will start in an

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

STINNER Victor added the comment: To see how long it takes to initialize urandom pool, you can grep your kernel logs. On my physical PC with real hardware, interruptions etc. it's quite fast: 5 seconds. -- Logs begin at mar. 2016-01-26 07:54:37 CET, ... ... juin 06 18:34:47 smithers kernel:

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Christian Heimes added the comment: Cory, thanks for summing it up for us. I totally agree with you. In my opinion it is troublesome to have different behavior on platforms. We can implement a workaround for Linux, but not for BSD. Or would O_NONBLOCK cause read() to fail with EWOULDBLOCK on

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Larry Hastings added the comment: So, in short, you don't know. #25003 is about Solaris, and the reporter clearly had the expectation that /dev/urandom would never block. The documentation on Linux is clear: /dev/urandom will never block. That's two. This "StackExchange" discussion:

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Cory Benfield added the comment: I have never seen it block in person, but I also wouldn't expect to. OS X's blocking guarantees are the same as FreeBSD's: that is, both /dev/random and /dev/urandom block until sufficient entropy has been gathered at startup and then never block again. This

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Larry Hastings added the comment: Are you certain that /dev/urandom will block on Mac OS X if sufficient entropy is not available? The dismissive tone ("this choice and distinction is not necessary") suggests that *their* implementation is superior, and it could hardly be seen as superior if

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Cory Benfield added the comment: > If you read #25003, it's clear that /dev/urandom is a well-known UNIX > facility with well-known, predictable behavior. One behavior that I'll draw > particular attention to now: it will never block. If the system is low on > entropy, /dev/urandom will

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Larry Hastings added the comment: I've revised my thinking on the subject. First, my previous statements stand: Python startup must not block. "import random" must not block. Now I'm thinking that "os.urandom()" must not block, too. Here's my reasoning. -- If you read #25003, it's clear

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Christian Heimes added the comment: Thanks Theodore! Your analysis was exactly what I was aiming for when I requested a thorough security analysis in form of a PEP. The correct choice of CPRNG is important for the overall security. I'm mostly concerned with the behavior of os.urandom(),

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Theodore Tso added the comment: I ran the experiment Colm asked me to run --- and yes, if you boot a system with Python 3.5.1 with the boot options "init=/usr/bin/python3", you're going to have a bad time. The problem is that in a KVM environment where things are very quiet, especially if

Re: [issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

On 07.06.2016 22:27, Theodore Tso wrote: > > Secondly, when I decided to add this behavior to getrandom(2), it was because > people were really worried that people would be using /dev/urandom for > security-critical things (e.g., initializing ssh host session keys, when > they'd _really_

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: Ted - I'd suggest the following to test. Boot an arbitrary Linux system with init=/usr/bin/python3 (assuming filesystems mounted etc). Python 3.5.1 (on Linux) will call getrandom() in blocking mode very early in its startup; if this happens before the pool is

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Donald Stufft added the comment: Thanks for weighing in Theodore, I think that matches what Colm's last suggestion was, and what I was personally OK with. To seed our SipHash function using GRND_NONBLOCK since it's likely that will be fine, and worst case we're just using it for some hash

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Theodore Tso added the comment: Hi. Colm alerted me to this bug, so I thought I would chime in as the author of Linux's getrandom(2) function. First of all, if you are OK with reading from /dev/urandom, then you might as well use getrandom's GRND_NONBLOCK flag. They are logically

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: I've spoken with Ted Ts'o (one advantage of working for Google) and taken a look in the Linux kernel source, and things are actually better than we'd feared. Firstly, calling getrandom() with GRND_NONBLOCK and a buffer size of less than or equal to 32 bytes

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Donald Stufft added the comment: > Once this has been done, it will never block again, and happily send you poor > random data if the entropy pool has been completely wiped of any entropy data > - without telling you. This doesn't actually happen in real life, once urandom has been

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Changes by Christian Heimes : -- nosy: -christian.heimes ___ Python tracker ___ ___

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Marc-Andre Lemburg added the comment: On 07.06.2016 21:12, Cory Benfield wrote: > >> Python hash randomization only happens once. So it's not a matter of how >> early we try the attack, it's a matter of how early we seed Python hash >> randomization. > > Sorry Larry, I was insufficiently

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Larry Hastings added the comment: I fear I may be changing my mind a little bit. However, I skipped breakfast--and now it's looking like a late lunch--so I simply have to step away for a while. Expect me to post in about two hours when I get some calories down and finally make up my tiny

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Donald Stufft added the comment: Colm, Great, then I think there's general agreement, we just need someone to review the nonblocking_urandom_noraise.patch (which my C is not strong enough to feel comfortable doing). That still leaves the `import random` issue, but I think we can reopen

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Cory Benfield added the comment: > Python hash randomization only happens once. So it's not a matter of how > early we try the attack, it's a matter of how early we seed Python hash > randomization. Sorry Larry, I was insufficiently clear (relying on context from earlier). I totally agree

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Larry Hastings added the comment: > This is only a DoS vector if you can hit the server so early in the boot > process that it doesn't have enough entropy. Python hash randomization only happens once. So it's not a matter of how early we try the attack, it's a matter of how early we seed

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Cory Benfield added the comment: > So you are intentionally accepting a new vector for DoS attacks, and calling this non-reduced security? This is only a DoS vector if you can hit the server so early in the boot process that it doesn't have enough entropy. The *second* enough entropy has been

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: Donald - Oh, that's interesting; thank you. I guess that system-crontab-generator is being called before that in the boot process. The most common trigger case, I guess, will then be cloud containers and VMs which are spun up for single applications. I think

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Donald Stufft added the comment: > specifically, early in the boot process, although not I think just on the > first boot of a system; Debian at least by default does not attempt to > preserve its entropy pool across a reboot.) Look at /etc/init.d/urandom in the initscripts package in Jessie

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: To clarify what the various patches do: 3.5.1 as released: os.urandom and hash secret initialization both attempt getrandom() in preference to reading /dev/urandom. Under certain circumstances, this will block, possibly indefinitely. Changeset 9de508dc4837:

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Matthias Klose added the comment: On 07.06.2016 16:47, Christian Heimes wrote: > > Christian Heimes added the comment: > > PSRT VETO! > > In the light of the upcoming release 3.5.2 I'm now putting on my PSRT hat > (Python Security Response Team) and proclaim a veto against any and all >

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Larry Hastings added the comment: That reminds me. I want to be clear: I think it's preferable that os.urandom() blocks when insufficient entropy is available. If Victor's patch changed that, it should be backed out. (Since non-blocking urandom is useful, perhaps in 3.6 os.urandom() should

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: Larry - To the first point: The combination of Victor's changeset 9de508dc4837 (based on my patch) and my most recent nonblocking_urandom_noraise patch (which is on top of 9de508dc4837) will do what you suggest for the hash secret initialization - ie: it is

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Donald Stufft added the comment: > As a final note, let me steer you towards this comment in Python/random.c: > > /* Issue #25003: Don' use getentropy() on Solaris (available since > * Solaris 11.3), it is blocking whereas os.urandom() should not block. */ > > Yes: we already had this

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Larry Hastings added the comment: Everybody: let's drop discussing "hashlib" unless someone says it actually is a problem. I think it was always, as we say in English, a "red herring". > The secret for SipHash is composed of two 64bit integers. The entire > _Py_HashSecret_t struct is 24

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Donald Stufft added the comment: I've attached a minimal patch for making it so ``import random`` does not block, it does this by changing what the default instance of Random() is seeded with, from os.urandom() to the time based fallback it currently employs. It does not change the behavior

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Christian Heimes added the comment: On 2016-06-07 19:36, Donald Stufft wrote: > > Donald Stufft added the comment: > >> Possible resolutions: >> * accept possible low-entropy initialization of the hash secret; using the >> patches supplied here by myself and Victor. >> * add a command-line

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Christian Heimes added the comment: On 2016-06-07 19:46, Larry Hastings wrote: > > Larry Hastings added the comment: > > Thank you for summarizing the debate. It made it a lot easier to > >> * blocking initialization of the hash secret. This occurs regardless of >> script contents; at

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Donald Stufft added the comment: > I don't follow whose patch does what. But here's what I find acceptable, > from a high level. > > * The semantics as presented by the documentation must be preserved. > os.urandom() and other operations that declare they're safe for cryptographic > use

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Larry Hastings added the comment: Thank you for summarizing the debate. It made it a lot easier to > * blocking initialization of the hash secret. This occurs regardless of > script contents; at present Python simply can't be used at all in low-entropy > situations. I feel that this issue is

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Donald Stufft added the comment: > Possible resolutions: > * accept possible low-entropy initialization of the hash secret; using the > patches supplied here by myself and Victor. > * add a command-line flag to disable "strong" initialization of the hash > secret (or revive the old -R flag).

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: Larry - I see at least two issues here, although they are related: * blocking initialization of the hash secret. This occurs regardless of script contents; at present Python simply can't be used at all in low-entropy situations. I feel that this issue is a

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: Donald - To be clear - no import of random or of hashlib is required to trigger this issue. The null script alone triggers the issue; the Python hash secret is initialized at startup regardless of script contents. Yes, there is a race condition at system boot

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Donald Stufft added the comment: > ISTM that the happy middle ground would be: > * seed the random module with non-cryptographically-secure random bits > * lazily seed hashlib I don't think it was actually hashlib that was causing the problem, but rather the initialization of SipHash, it just

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Donald Stufft added the comment: > Please note: this is *not* just a VM/cloud issue. This is observed on > physical standalone systems. But it should only occur on initial boot I believe? AFAIK all of the major linux vendors have stored a seed file once the machine has been booted and the

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Larry Hastings added the comment: > PSRT VETO! This is an amusing concept, but membership in the PSRT does not empower you with a "veto". On the other hand, being Release Manager does give me some say here. > You wouldn't add a workaround for broken CPU instructions to math.c or >

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: ... the script is /lib/systemd/system-generators/systemd-crontab-generator, although that's not hugely germane to the discussion. Arranging for PYTHONHASHSEED to be set while it's called wouldn't be impossible of course, although a command-line flag would be

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: Christian - Please note: this is *not* just a VM/cloud issue. This is observed on physical standalone systems. The issue (on Debian) is that the Python script is called very early in the boot process; in particular before most hardware initialization is

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Marc-Andre Lemburg added the comment: On 07.06.2016 16:59, Christian Heimes wrote: > > Christian Heimes added the comment: > > On 2016-06-07 16:49, Marc-Andre Lemburg wrote: >> This gives people a clear choice and doesn't cause people >> to have to reconsider using the random module or wait

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Christian Heimes added the comment: On 2016-06-07 16:49, Marc-Andre Lemburg wrote: > This gives people a clear choice and doesn't cause people > to have to reconsider using the random module or wait for > Python hash randomization to initialize itself when using > Python during

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: Christian - I would like to make one further comment: The only reason getrandom() was used instead of /dev/random was to avoid wasting a file descriptor. The previous behavior was in use for many years with no security issues; it was changed for FD

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Marc-Andre Lemburg added the comment: On 07.06.2016 16:18, Donald Stufft wrote: > > Donald Stufft added the comment: > >> I just don't understand why you are apparently not willing to even consider >> compromises. > > I have one thing that I hold immutable here, That os.urandom should use

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Christian Heimes added the comment: PSRT VETO! This ticket is turning into a bike-shedding discussion. In the light of the upcoming release 3.5.2 I'm now putting on my PSRT hat (Python Security Response Team) and proclaim a veto against any and all changes to os.urandom(). The security

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: Victor - I see three options for 3.5.2: * continue with the 3.5.1 behaviour, which blocks all python invocations in low-entropy situations. I think this is highly undesireable. * apply my patches, which fixes the hash secret initialization but not 'import

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: The attached patch (against tip) is a very quick attempt to implement the desired behavior: * add a "nonblocking" argument to py_getrandom() * dev_urandom_python() sets this to 0, to request blocking operation * dev_urandom_noraise() sets this to 1 to request

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Stefan Krah added the comment: man urandom: "A read from the /dev/urandom device will not block waiting for more entropy. As a result, if there is not sufficient entropy in the entropy pool, the returned values are theoretically vulnerable to a cryptographic attack on the

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Donald Stufft added the comment: > I just don't understand why you are apparently not willing to even consider > compromises. I have one thing that I hold immutable here, That os.urandom should use the best interfaces provided by the OS to ensure that it always returns cryptographically

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

STINNER Victor added the comment: > nonblocking_urandom_noraise.patch doesn't fix the issue #21470. Sorry, it's the issue #25420: "import random" blocks on entropy collection on Linux with low entropy -- ___ Python tracker

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

STINNER Victor added the comment: nonblocking_urandom_noraise.patch doesn't fix the issue #21470. -- ___ Python tracker ___

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Christian Heimes added the comment: I'm with Donald here. Python must not reduce security just for a special case. It doesn't mean that we should not address and fix this special case -- just treat it as special. 1) For your use case, the hash randomization key for the SipHash PRN doesn't

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Donald Stufft added the comment: > I have no objection to *deliberate* invocations of the system RNG blocking if > needed. Presumably this behavior can be codified into the various APIs. > > My objection is *entirely* to _PyRandom_Init() calling a potentially-blocking > RNG source, before

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Marc-Andre Lemburg added the comment: On 07.06.2016 15:21, Donald Stufft wrote: > > The patch causes a regression because I'm relying on the 3.5 behavior of > getting secure randomness from ``os.urandom`` via the ``getrandom()`` system > call (behavior that was documented in the Whats New in

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: I have no objection to *deliberate* invocations of the system RNG blocking if needed. Presumably this behavior can be codified into the various APIs. My objection is *entirely* to _PyRandom_Init() calling a potentially-blocking RNG source, before script parsing

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Donald Stufft added the comment: > With the greatest respect, you're talking about introducing multi-minute > delays into the startup times of hundreds of millions of systems, regardless > of whether they have a proximate requirement for cryptographically-secure RNG > sources. No I'm not.

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Alex Gaynor added the comment: Colm -- how is that situation not addressed by fixing the hash seed generation specifically, rather than patching all consumers of os.urandom? -- ___ Python tracker

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

Colm Buckley added the comment: Donald - With the greatest respect, you're talking about introducing multi-minute delays into the startup times of hundreds of millions of systems, regardless of whether they have a proximate requirement for cryptographically-secure RNG sources. I don't think

  1   2   >