[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[STINNER Victor]

STINNER Victor added the comment:

> Re-reviewing your patch, I agree the What's New comment is still accurate.

Thanks for double checking ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Nick Coghlan]

Nick Coghlan added the comment:

Sorry, with all the different proposals kicking around, I somehow got the 
impression we'd reverted entirely to just reading from /dev/urandom without 
ever using the new syscall.

Re-reviewing your patch, I agree the What's New comment is still accurate.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[STINNER Victor]

STINNER Victor added the comment:

Nick Coghlan: "One last fix needed to fully revert this is to remove the 
mention from the Python 3.5 What's New documentation: 
https://docs.python.org/3.5/whatsnew/3.5.html#os;

This sentence?

"The urandom() function now uses the getrandom() syscall on Linux 3.17 or 
newer, and getentropy() on OpenBSD 5.6 and newer, removing the need to use 
/dev/urandom and avoiding failures due to potential file descriptor exhaustion."

Why removing it? It's still correct that getrandom() is used by os.urandom() in 
the common case.

The corner case (urandom entropy pool not initialized) is already documented 
(including a "Changed in version 3.5.2: ..."):
https://docs.python.org/3.5/library/os.html#os.urandom

I don't think that it's worth to mention the corner case in What's New in 
Python 3.5.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Nick Coghlan]

Nick Coghlan added the comment:

One last fix needed to fully revert this is to remove the mention from the 
Python 3.5 What's New documentation: 
https://docs.python.org/3.5/whatsnew/3.5.html#os

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

Colm Buckley: I've read the code, *and* stepped through it, and AFAICT it is no 
longer even possible for Python on Linux to call getrandom() in a blocking way. 
 Thanks for doing this!  I'm marking the issue as closed.

--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

I just posted to python-dev and asked Guido to make a BDFL ruling.  I only 
represented my side, both because I worried I'd do a bad job of representing 
*cough* literally everybody else *cough*, and because it already took me so 
long to write the email.  All of you who disagree with me, I'd appreciate it if 
you'd reply to my python-dev posting and state your point of view.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

> Regardless of the behavior of os.urandom (and 'import random'), is it agreed 
> that the current state of _PyRandom_Init is acceptable for 3.5.2?

I'll get back to you with a specific yes or no.  What I want is that it the 
behavior removed where "import random" can block unboundedly on Linux because 
it's waiting for the entropy pool to fill.  If the code behaves like that, then 
yes, but I'm not giving it my official blessing until I read it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Nick Coghlan]

Changes by Nick Coghlan :


--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[STINNER Victor]

STINNER Victor added the comment:

> The current behavior (as of 9de508dc4837) is that it will never block on 
> Linux, but could still block on other OS if called before /dev/urandom is 
> initialized.

In practice, only Linux is impacted. See the rationale:
https://haypo-notes.readthedocs.io/pep_random.html#scope-of-the-python-blocks-at-startup-issue


> We have not determined a satisfactory solution for other operating systems.

Stp. This issue is specific to Linux. If you want to fix the issue on other 
operating systems, please open a new issue.

Oh, you know what? I already opened such issue :-) The issue #27266 wants to 
fix the issue on all platforms, not only Linux. Open a second issue if you 
prefer.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[STINNER Victor]

STINNER Victor added the comment:

I opened the issue #27272: "random.Random should not read 2500 bytes from 
urandom".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

Larry -

Regardless of the behavior of os.urandom (and 'import random'), is it agreed 
that the current state of _PyRandom_Init is acceptable for 3.5.2?

The current behavior (as of 9de508dc4837) is that it will never block on Linux, 
but could still block on other OS if called before /dev/urandom is initialized. 
We have not determined a satisfactory solution for other operating systems. 
Note that no other OS have reported a problem 'in the wild', probably because 
of their extreme rarity in VM/container environments and the lack of Python in 
their early init sequence.

Colm

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[ppperry]

Changes by ppperry :


--
title: Python 3.5 running on Linux kernel 3.17+ can block at startup or on 
importing /arguinthe random module on getrandom() -> Python 3.5 running on 
Linux kernel 3.17+ can block at startup or on importing the random module on 
getrandom()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing /arguinthe random module on getrandom()

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 08.06.2016 22:49, Larry Hastings wrote:
> 
> Third, because the os module is in general a thin wrapper over what the OS 
> provides, I disapprove of "cryptorandom()" and "pseudorandom()" going into 
> the os module.  There are no functions with these names on any OS of which 
> I'm aware.  This is why I proposed "os.getrandom(n, block=True)".  From its 
> signature, the function it calls on your OS will be obvious, and its 
> semantics on your OS will be documented by your OS.
> 
> Thus I am completely unwilling to add os.cryptorandom() and os.pseudorandom() 
> in 3.5.2.

That was a sketch for 3.6 to resolve the ambiguity between the
different use cases.

You're right, it's better to move such things to the random
module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing /arguinthe random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

I am increasingly convinced that I'm right.

--

First, consider that the functions in the os module, as a rule, are a thin 
shell over the equivalent function provided by the operating system.  If Python 
exposes a function called os.XYZ(), and it calls the OS, then with few 
exceptions it does so by calling a function called XYZ().**

This has several ramifications, and these are effectively guarantees for the 
Python programmer:

* You can read your local man pages (or equivalent) to see how the function 
behaves oh your system.  Python occasionally improves on the functionality 
provided; os.utime() provides a lot more functionality than POSIX utime.  But 
it never *degrades* the functionality provided.

* It's implied, and strongly preferred, that the function is atomic: it will 
make exactly one system call.  I once proposed simulating behavior for an os 
module function using a series of system calls, and this approach was rejected 
because it wasn't atomic.  So if you see a function os.XYZ(), you may predict 
that Python will call XYZ() exactly once, and with only a few exceptions you'll 
be right.

Now read this snippet of the documentation for os.urandom():

"The returned data should be unpredictable enough for cryptographic 
applications, though its exact quality depends on the OS implementation. On a 
Unix-like system this will query /dev/urandom, and on Windows it will use 
CryptGenRandom()."

That text has been in the documentation for os.urandom() since at least Python 
2.6.  (That's as old as we have on the web site; I didn't go hunting for older 
documentation.)

Thus the documentation for os.urandom():

* explicitly says it uses /dev/urandom, and

* explicitly *does not* guarantee cryptographic strength random numbers on all 
platforms at all times.

Thus, while it's laudable to try and give the user higher-quality random bits 
when they call os.urandom(), you cannot degrade the behavior of the system's 
/dev/urandom when doing so.  On Linux /dev/urandom is *guaranteed* to never 
block.  This guarantee is so strong, Mr. Ts'o had to add a separate facility to 
Linux (/dev/random) to permit blocking.  os.urandom() *must* replicate this 
behavior.

What I'm proposing is that os.urandom() may use getrandom(RND_NOBLOCK) to 
attempt to get higher-quality random bits, but it *must not block*.  If it 
fails, it will use /dev/urandom, *exactly as it is documented to do*.

(Naturally this flunks the "atomic operation" test.  But in the case of 
procuring random bits, the atomicity of its operation is obviously irrelevant.)


** The exception to this, naturally, is Windows.  Internally the os module is 
called "posixmodule"--and this is no coincidence.  AFAIK every platform 
supported by CPython is POSIX-based except Windows.  The choice was made long 
ago to simulate POSIX behavior on Windows so as to present a consistent API to 
the programmer.  If you're curious about this, and have the time, read the 
implementation of os.stat for Windows.  What a rush!

--

Second, I invoke the "consenting adults" rule.  Python provides well-documented 
behavior for os.urandom().  You cannot make assumptions about the use case of 
the caller and decide for them that they would prefer the function block in an 
unbounded fashion rather than provide low-quality random bits.

And yes, unbounded.  As covered earlier in the thread, it only blocked for 90 
seconds before systemd killed it.  We don't know how long it would actually 
have blocked.  This is completely unacceptable--for startup, for "import 
random", and for "os.urandom()" on Linux.

--

Third, because the os module is in general a thin wrapper over what the OS 
provides, I disapprove of "cryptorandom()" and "pseudorandom()" going into the 
os module.  There are no functions with these names on any OS of which I'm 
aware.  This is why I proposed "os.getrandom(n, block=True)".  From its 
signature, the function it calls on your OS will be obvious, and its semantics 
on your OS will be documented by your OS.

Thus I am completely unwilling to add os.cryptorandom() and os.pseudorandom() 
in 3.5.2.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing /arguinthe random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

@pitti -

We already discussed this; there are cases where it's not practical to set an 
environment variable. The discussion eventually converged on "it is not 
desirable that Python should block on startup, regardless of system RNG status".

Re: the triggering bug; it was actually 
/lib/systemd/system-generators/systemd-crontab-generator (in systemd-cron) 
which caused the behavior to be noticed in Debian. It wasn't a change in 
systemd behavior, per se (that has been a Python script for some time), it was 
the fact that it was being called before the system PRNG had been initialized. 
With the change from /dev/urandom to getrandom() in 3.5.1, this caused a 
deadlock at boot.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing /arguinthe random module on getrandom()

[Martin Pitt]

Martin Pitt added the comment:

> you could give some kind of command-line flag

That already exists -- set PYTHONHASHSEED=0.

> But I'll let someone else have the joys of negotiating with Lennart, and I 
> won't blame the Python devs if using GRND_NONBLOCK unconditionally is less 
> painful than having to work with the systemd folks. 

In case it's of any relief: This has nothing to do with having to change 
anything in systemd itself -- none of the services that systemd ships use 
Python. The practical case where this bug appeared was cloud-init (which is 
written in Python), and that wants to run early in the boot sequence even 
before the network is up (so that tools like "pollinate" which gather entropy 
from the cloud host don't kick in yet). So if there's any change needed at all, 
it would be in cloud-init and similar services which run Python during early 
boot.

--
nosy: +pitti
title: Python 3.5 running on Linux kernel 3.17+ can block at startup or on 
importing the random module on getrandom() -> Python 3.5 running on Linux 
kernel 3.17+ can block at startup or on importing /arguinthe random module on 
getrandom()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[STINNER Victor]

STINNER Victor added the comment:

I spent almost my whole day to read this issue, some related issues, and some 
more related links. WOW! Amazing discussing. Sorry that Christian decided to 
quit the discussion (twice) :-(

Here is my summary: http://haypo-notes.readthedocs.io/pep_random.html

tl; dr "The issue is to find a solution to not block Python startup on such 
case, and keep getrandom() enhancement for os.urandom()."

--

Status of Python 3.5.2: 
http://haypo-notes.readthedocs.io/pep_random.html#status-of-python-3-5-2

My summary: "With the changeset 9de508dc4837: Python doesn’t block at startup 
anymore (issues #25420 and #26839 are fixed) and os.urandom() is as secure as 
Python 2.7, Python 3.4 and any application reading /dev/urandom."

=> STOP! don't touch anything, it's now fine ;-) (but maybe follow my link for 
more information)

--

To *enhance* os.urandom(), always use getrandom() syscall on Linux, I opened 
the issue #27266. I changed the title to "Always use getrandom() in os.random() 
on Linux and add block=False parameter to os.urandom()" to make my intent more 
explicit.

As some of you have already noticed, it's not easy to implement this issue! 
There are technical issues to implement os.urandom(block=False).

In fact, this issue tries to fix two different but close issues:

(a) Always use getrandom() for os.urandom() on Linux
(b) Implement os.urandom(block=False) on *all* platforms

The requirement for (a) is to not reopen the bug #25420 (block on "import 
random"). dstufft proposed no-urandom-by-default.diff (attached to this issue), 
but IMHO it makes the random module worse than before. I proposed (b) as the 
correct fix. It's a work-in-progress, please come on the issue #27266 to help 
me!

--

Please contact me if we want to fix/enhance my doc 
http://haypo-notes.readthedocs.io/pep_random.html

Right now, I'm not interested to convert this summary to a real PEP. It looks 
like you agree on solutions. We should now invest our time on solutions rather 
than listing again all issues ;-)

I know that it's really hard, but I suggest to abandon this issue (since, 
again, it's closed!), and focus on more specific issues and work on fixing 
issues. No? What do you think?

--

IMHO The problem in this discussion is that it started with a very well defined 
issue (Python blocks at startup on Debian Testing in a script started by 
systemd when running in a VM) to a wide discussion about all RNG, all kinds of 
issues related to RNG and a little bit to security in general.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

Just to re-state; I think we have three problems:

1) _Py_HashSecret initialization blocking. Affects all Python invocations; 
already a substantial issue on Debian testing track (90s startup delay).

* there seems to be general agreement that this does not need a 'strong' secret 
in a script called at/near startup.
* On Linux, getrandom(GRND_NONBLOCK) *or* /dev/urandom are sufficient for this 
initialization.
* On other OS, we don't have a non-blocking kernel PRNG; this is probably not 
an issue for Solaris or OS X, and only a possible issue for OpenBSD.
* Is it acceptable to fall back to an in-process seed generation for the cases 
where initialization via /dev/urandom fails (NB : there have been no reports of 
this type of failure in the wild).

* existing tip with or without nonblocking_urandom_noraise.patch addresses this 
for Linux. Solution for other OS remains to be written.
* Possibly can be considered non-blocking for other OS, as there has been no 
recent regression in behavior.

2) Blocking on 'import random' and/or os.urandom. I don't see a clear consensus 
on the Right Thing for this case. Existing tip (without 
nonblocking_urandom_noraise.patch) addresses it for Linux, but solution is not 
universally accepted. Unclear whether this is a 3.5.2 blocker.

3) Design of future APIs for >= 3.6. The most frequent suggestion is something 
like os.pseudorandom() (guaranteed nonblocking) and os.cryptorandom() 
(guaranteed entropy); I guess this needs to go to the dev list for full 
discussion - is it safely out of scope for this bug?

My suggestion (for what it's worth): accept Victor's changeset plus 
nonblocking_urandom_noraise for 3.5.2 (I'll submit a proper patch shortly), 
recommend userspace workarounds for the blocking urandom issue, propose new 
APIs for 3.6 on the dev list.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[STINNER Victor]

STINNER Victor added the comment:

>> The current behavior is that Python *will not start at all* if getrandom() 
>> blocks (because the hash secret initialization fails).
> It starts jsut fine, it just can possible takes awhile.

In my experience, connecting to a VM using SSH with low entropy can take longer 
than 1 minute. As an user, I considered that the host was down. Longer than 1 
minute is simply too long.

It's unclear to me if getrandom() can succeed (return random bytes) on embedded 
devices without hardware RNG. Can it take longer than 1 minute?

Is it possible that getrandom() simply blocks forever?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

Victor -

Yes, it is possible for it to block forever - see the test I proposed for Ted 
upthread. The triggering case (systemd-crontab-generator) delays for 90 
seconds, but is *killed* by systemd after that time; it doesn't itself time out.

Colm

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Theodore Tso]

Theodore Tso added the comment:

Oh --- and about people wondering whether os.random is being used for 
cryptographic purposes or not "most of the time" or not --- again, welcome to 
my world.  I get complaints all the time from people who try to do "dd 
if=/dev/urandom of=/dev/hdX bs=4k" and then complain this is too slow.

Creating an os.cryptorandom and os.pseudorandom might be a useful way to go 
here.  I've often considered whether I should create a /dev/frandom for the 
crazies who want to use dd as a way to wipe a disk, but to date I've haven't 
thought it was worth the effort, and I didn't want to encourage them.  Besides, 
isn't obviously the right answer is to create a quickie python script?  :-)

Splitting os.random does beg the question of what os.random should do, however. 
 If you go down that path, I'd suggest defaulting to the secure-but-slow choice.

I'd also suggest assuming it's OK to put the onus on the people who are trying 
to run python scripts during early boot to have to either add some command 
flags to the python interpreter, or to otherwise make adjustments, as being 
completely fair.  But again, that's my bias, and if people don't want to deal 
with trying to ask the systemd folks to make a change in their code, I'd 
_completely_ understand.

My design preference is that outside of boot scripts, having os.random block in 
the same of security is completely fair, since in that case you won't deadlock 
the system.  People of good will may disagree, of course, and I'm not on the 
Python development team, so take that with whatever grain of salt you wish.   
At the end of the day, this is all about tradeoffs, and you know your 
customer/developer base better than I do.\

Cheers!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Theodore Tso]

Theodore Tso added the comment:

One of the reasons why trying to deal with randomness is hard is because a lot 
of it is about trust.  Did Intel backdoor RDRAND to help out the NSA?   You 
might have one answer if you work for the NSA, and perhaps if you are willing 
to assume the worst about the NSA balancing its equities between its signals 
intelligence mission and providing a secure infrastructure for its customers 
and keeping the US computing industry strong.   Etc., etc.

It is true that OS developers are trying to make their random number generators 
be initialized more quickly at boot time.  Part of this is because of the 
dynamic which we can all see at work on the discussion of this bug.  Some 
people care about very much about not blocking; some people want Python to be 
useful during the boot sequences; some people care very much about security 
above all else; some people don't trust application programmers.  (And if you 
fit in that camp; congratulations, now you know how I often feel when I worry 
about user space programmers doing potentially crazy things and I have no way 
of even knowing about them until the security researchers publish a web site 
such as http://www.factorable.net)

>From the OS's perspective, one of the problems is that it's very hard to know 
>when you have actually achieved a securely initialized random number 
>generator.  Sure, we can say we've done this once we have accumulated at least 
>128 bits of entropy, but that begs the question of when you've collected a bit 
>of entropy.  There's no way to know for sure.  On current systems, we assume 
>that each interrupt gathers 1/64th of a bit of entropy on average.  This is an 
>incredibly conservative number, and on real hardware, assuming the normal 
>bootup activity, we achieve that within about 5 seconds (plus/minus 2 seconds) 
>after boot.   On Intel, on real hardware, I'm comfortable cutting this to 1 
>bit of entropy per interrupt, which will speed up things considerably.  In an 
>ARM SOC, or if you are on a VM and you don't trust the hypervisor so you don't 
>use virtio-rng, is one bit of entropy per interrupt going to be good enough?  
>It's hard to say.

On the other hand, if we use too conservative a number, there is a risk that 
userspace programmers (such as some have advocated on the discussionon this 
bug) to simply always use GRND_NONBLOCK, or fall back to /dev/urandom, and then 
if there's a security exposure, they'll cast the blame on the OS developers.  
The reality is that we really need to work together, because the real problem 
are the clueless people writing python scripts at boot time to create long-term 
RSA private keys for IOT devices[1].  :-)

So when people assert that developers at FreeBSD are at work trying to speedup 
/dev/random initialization, folks need to understand that there's no magic 
here.  What's really happening is that we're all trying to figure out which 
defaults work the best.  In some ways the FreeBSD folks have it easier, because 
they support a much fewer range of platforms.  It's a lot easier to get things 
right on x86, where we have instructions like RDTSC and RDRAND to help us out.  
It's a lot harder to be sure you have things right for ARM SOC's.   There are 
other techniques such as trying to carry entropy over from previous boot 
sessions, but (a) this requires support from the boot loaders, and on an OS 
with a large number of architectures, that means adding support to a large 
number of different ways of booting the kernel --- and it doesn't solve the 
"consumer device generating keys after a cold start when the device is freshly 
removed from the packaging".

As far as adding knobs, such as "blocking vs non-blocking", etc., keep in mind 
that as you add knobs, you increase the knowledge of the system that you force 
onto the next layer of the stack.  So this goes to the question of whether you 
trust application programmers will be able to get things right.

So Ted, why does Linux expose /dev/random vs /dev/urandom?  Historical reasons; 
some people don't believe that relying on cryptogaphic random number generators 
is sufficient, they *want* to use entropy which has minimal reliance on the 
belief that NSA ***probably*** didn't leave a back door into SHA-1, for 
example.  It is for that reason that /dev/random exists.  These days, the 
number of people who believe that to be true are very small, but I didn't want 
to make changes in existing interfaces.  For similar reasons I didn't want to 
suddenly make /dev/urandom block.   The fact that getrandom(2) blocks only 
until the cryptographic RNG has been initialized, and that it depends on a 
cryptogaphic RNG, is the consensus that *most* people have come to, and it 
reflects my recommendations that unless you ***really*** know what you are 
doing, the right thing to do is to call getrandom(2) with the flags field set 
to zero, and to be happy.   Of course, many people are sure they know what they 

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[STINNER Victor]

STINNER Victor added the comment:

Cory Benfield (msg267637): "if the purpose of this patch was to prevent long 
startup delays, *it failed*. On all the systems above os.urandom may continue 
to block system startup."

I don't pretend fixing the issue on all operating systems. As stated in the 
issue title, this issue is specific to Linux. I understand that the bug still 
exists on other platforms and is likely to require a specific fix for each 
platform.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[STINNER Victor]

STINNER Victor added the comment:

Donald: "Cory wasn't speaking about (non)blocking in general, but the case 
where (apparently) it's desired to not block even if that means you don't get 
cryptographically secure random in the CPython interpreter start up. (...)"

Oh sorry, I misunderstood his message.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Christian Heimes]

Christian Heimes added the comment:

I'm unsubscribing from this ticket for the second time. This form of discussion 
is becoming toxic for me because I strongly beliefe that it is not the correct 
way to handle a security-related problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Christian Heimes]

Changes by Christian Heimes :


--
nosy:  -christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

Even though it may sound like a minor problem that os.urandom()
blocks on startup, I think the problem is getting more and more
something to consider given how systems are used nowadays.

Today, we no longer have the case where you keep a system up
and running for multiple years as we had in the past. VM,
containers and other virtualizations are spun up and down at
a high rate, so the boot cycle becomes more and more important.

FreeBSD, for example, is also concerned about the blocking issue
they have in their implementation:

https://wiki.freebsd.org/201308DevSummit/Security/DevRandom

and they are trying to resolve this by making sure to add as
much entropy they can find very early on in the process.

Now, most applications you run early on in the boot process
are not going to be applications that need crypto random
numbers and this is where I think the problem originates.

We've been telling everybody to use os.urandom() for seeding,
and so everyone uses it, including many many applications that
don't even require crypto random seeding.

The random module is the perfect example.

Essentially, we'd need to educate people that there's a difference
in requesting crypto random data and pseudo random data.

While we can fix the the cases in the stdlib and
the interpreter that don't need crypto random data to use
other means of seeding (e.g. reading straight from /dev/urandom
on Linux or gathering other data to mix into a seed), existing
applications out there will continue to use os.urandom() for
things that don't need crypto random numbers - after all, we told
them to use it.

Some of these will eventually be hit by the blocking problem,
even for applications such as Monte Carlo simulations that
don't need crypto random and should thus not have to wait for
some entropy pool to get initialized.

Now, applications that do need crypto random data should be
able to request this from Python via the stdlib and os.urandom()
may sound like a good basis, but since this is designed as
interface to /dev/urandom, it doesn't block on Linux, so
not such a good choice.

Using /dev/random probably doesn't work either, because this can
block unexpected even after initialization.

IMO, the best way forward and to educate application writers
about the problems is to introduce a two new APIs in 3.6:

os.cyptorandom() for getting OS crypto random data
os.pseudorandom() for getting OS pseudo random data

Crypto applications will then clearly know that
os.cryptorandom() is the right choice for them and
everyone else can use os.pseudorandom().

The APIs would on Linux and other platforms then use getrandom()
with appropriate default settings, i.e. blocking or raising
for os.cryptorandom() and non-blocking, non-raising for
os.pseudorandom().

As for the solving the current issue, we will have to
give people some way to get at non-blocking pseudo random data,
if they need it early in the boot process. With the
proposed change, this is still possible via reading
/dev/urandom directly on Linux, so not everything is
lost.

BTW: Wikipedia has a good overview of how the different
implementations of /dev/random work across platforms:

https://en.wikipedia.org/wiki//dev/random

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Donald Stufft]

Donald Stufft added the comment:

Larry,

I would greatly prefer it if we would allow os.urandom to block on Linux, at 
least by default. This will make os.urandom behavior similarly on most modern 
platforms. The cases where this is going to matter are extreme edge cases, for 
most users they'll just silently be a bit more secure-- important for a number 
of use cases of Python (think for instance, if someone has a SSH server written 
in Twisted that generates it's own host keys, a perfectly reasonable use of 
os.urandom). We've been telling people that os.urandom is the right source for 
generating randomness for cryptographic use for ages, and I think it is 
important to use the tools provided to us by the platform to best satisfy that 
use case by default-- in this case, getrandom() in blocking mode is the best 
tool provided by the Linux platform.

People writing Python code cannot expect that os.urandom will not block, 
because on most platforms it *will* block on intialization. However, the cases 
where it will block are a fairly small window, so by allowing it to block we're 
giving a better guarantee for very little downside-- essentially that something 
early on in the boot process shouldn't call os.urandom(), which is the right 
behavior on Linux (and any other OS) anyways.

The problem is that the Python interpreter itself (essentially) calls 
os.urandom() as part of it's start up sequence which makes it unsuitable for 
use in very early stage boot programs. In the abstract, it's not possible to 
fix this on every single platform without removing all use of os.urandom from 
Python start up (which I think would be a bad idea). I think Colm's 
nonblocking_urandom_noraise.patch is a reasonable trade off (perhaps not the 
one I would personally make, but I think it's reasonable). If we wish to ensure 
that Python interpreter start up never blocks on Linux without needing to 
supply any command line flags or environment variables, then I would strongly 
urge us to adopt his patch, but allow os.urandom to still block.

In other words, please let's not let systemd's design weaken the security 
guarantees of os.urandom (generate cryptographically secure random bytes using 
the best tools provided by the platform). Let's make a targeted fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Christian Heimes]

Christian Heimes added the comment:

On 2016-06-08 11:39, STINNER Victor wrote:
> 
> STINNER Victor added the comment:
> 
> no-urandom-by-default.diff uses a very weak source of entropy for 
> random.Random :-( I'm fighting against weak sources of entropy since many 
> years...

It is totally fine to init the MT of random.random() from a weak entropy
source. Just keep in mind that a Mersenne Twister is not a CPRNG. There
is simply no reason why you want to init a MT from a CPRNG.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[STINNER Victor]

STINNER Victor added the comment:

Larry Hastings:

> > Hi. I created the issue #27266 "Add block keyword-only optional parameter 
> > to os.urandom()" (...)
> > Happy?

> Probably not. (...)

I replied on the issue #27266. Sorry I'm unable to follow this issue, there are 
too many messages now :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[STINNER Victor]

STINNER Victor added the comment:

no-urandom-by-default.diff uses a very weak source of entropy for random.Random 
:-( I'm fighting against weak sources of entropy since many years...

This change introduces the bug similar to OpenSSL RAND_bytes() bug (two 
processes with the same pid can produce the same random sequence): two Python 
processes started "at the same time" (with a resolution of 1/256 sec ~= 3.9 ms) 
produces the same random sequence.

With my script:
---
import subprocess, sys
args = [sys.executable, '-S', '-c', 'import random; print([random.randint(0, 
999) for _ in range(4)])']
numbers = set()
procs = [subprocess.Popen(args, stdout=subprocess.PIPE) for _ in range(10)]
for proc in procs:
stdout = proc.communicate()[0]
numbers.add(stdout.rstrip())
for line in numbers:
print(line.decode())
print("duplicates", len(procs) - len(numbers))
---

Output:
---
[68, 812, 821, 421]
[732, 506, 562, 439]
[70, 711, 476, 230]
[411, 474, 729, 837]
[530, 161, 699, 521]
[818, 897, 582, 38]
[42, 132, 359, 275]
[630, 863, 370, 288]
[497, 716, 61, 93]
duplicates 1
---

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

> Hi. I created the issue #27266 "Add block keyword-only optional parameter
> to os.urandom()" which is compromise between all proposed solutions and
> should fix *all* urandom issues ;-)
> 
> * os.urandom() remains secure by default, as asked by our security experts
> * Python startup (hash secret) and "import random" don't block anymore
> 
> Happy?

Probably not.

What is the default value of the "block" parameter?

If called with block=False on FreeBSD, where /dev/urandom may block sometimes, 
what does the function do?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Cory Benfield]

Cory Benfield added the comment:

> You're right, it's remotely possible that on platforms where /dev/urandom
> could block, Python startup could therefore also block.  And I'm not
> proposing we fix that, as so far nobody has reported it as a problem.
> 
> This suggests to me that yes I'm talking specifically about the regression
> on Linux in the 3.5 series.

Ok, so with that clarification I personally would prefer Victor's patch from 
#27266, but can also understand wanting to leave the codebase as-is. Either way 
would be consistent with your goals, Larry. Victor's patch is more secure, but 
does cause os.urandom to diverge from the semantics of /dev/urandom in extreme 
conditions (specifically, early boot) on Linux.

That's your tradeoff to make, Larry. =) I think both sides have been 
well-argued here. Thanks for clarifying.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

You're right, it's remotely possible that on platforms where /dev/urandom could 
block, Python startup could therefore also block.  And I'm not proposing we fix 
that, as so far nobody has reported it as a problem.

This suggests to me that yes I'm talking specifically about the regression on 
Linux in the 3.5 series.

But honestly it's too late for me to say for sure one way or another.  I need 
to go to bed.

p.s if we have to slip RC1 by a day or two in order to get this settled, that's 
okay, but hopefully I can keep final on schedule.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[STINNER Victor]

STINNER Victor added the comment:

Hi. I created the issue #27266 "Add block keyword-only optional parameter to 
os.urandom()" which is compromise between all proposed solutions and should fix 
*all* urandom issues ;-)

* os.urandom() remains secure by default, as asked by our security experts
* Python startup (hash secret) and "import random" don't block anymore

Happy?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Cory Benfield]

Cory Benfield added the comment:

> What I'm trying to avoid here is the surprising situation where someone is 
> using Python on a system where /dev/urandom will never block, and 
> os.urandom() blocks.

At this point I literally do not understand what issue we're trying to solve 
then.

If the problem is that os.urandom() must behave exactly like /dev/urandom, then 
sure, that got regressed.

However, I've been talking based on your two previous pronouncements:

> First, my previous statements stand: Python startup must not block.  "import 
> random" must not block.

and

> I am officially making a pronouncement as Release Manager: Python 3.5 *must
> not* take 90 seconds to start up under *any* circumstances.  I view this as
> a performance regression, and it is and will remain a release blocker for
> 3.5.2.
> 
> Python *must not* require special command-line flags to avoid a 90 second
> startup time.  Python *must not* require a special environment-variable to
> avoid a 90 second startup time.  This is no longer open to debate, and I
> will only be overruled by Guido.

Now, if that's the case, then this patch does not fix that problem. It fixes 
that problem *on Linux*, but not on BSDs.

Perhaps you meant to say that those pronouncements only apply to Linux. That's 
fine, it's your prerogative. But as written, they don't: they're unconditional. 
And if they are unconditional, then again I feel like we have to say that 
/dev/urandom should get *out* of the call path on interpreter startup, because 
it absolutely can block. And based on Colm's original problem around gathering 
entropy, which is almost certainly not a Linux-specific concern, I see no 
reason to believe that this is a hypothetical concern on the BSDs.

So, let me ask a very direct question: does the position about 90s startup 
apply only to Linux?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

I don't know if anyone literally still uses BSD.  But on FreeBSD, /dev/urandom 
can block.

So let me revise my statement slightly.  Developers on platform X know how 
*their* /dev/urandom behaves.  They should rightly expect that os.urandom() is 
a thin wrapper around their local /dev/urandom.  If their /dev/urandom doesn't 
block, then os.urandom() shouldn't block.  If their /dev/urandom blocks, then 
it's acceptable that their os.urandom() would block.

What I'm trying to avoid here is the surprising situation where someone is 
using Python on a system where /dev/urandom will never block, and os.urandom() 
blocks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

[[ Larry - thanks for the Mercurial pointers. I was starting from the Debian 
sources because I initially assumed this was a Debian problem. Will switch to 
hg in future. ]]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Christian Heimes]

Christian Heimes added the comment:

Larry,  /dev/urandom blocks on BSD when it hasn't been seeded yet. But it looks 
like we can use sysctl to fetch the seed state from kern.random.sys.seeded.

https://www.freebsd.org/cgi/man.cgi?query=random=4

 The software generator will start in an unseeded state, and will block
 reads until it is (re)seeded.  This may cause trouble at system boot when
 keys and the like are generated from /dev/random so steps should be taken
 to ensure a reseed as soon as possible.  The sysctl(8) controlling the
 seeded status (see below) may be used if security is not an issue or for
 convenience during setup or development.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[STINNER Victor]

STINNER Victor added the comment:

To see how long it takes to initialize urandom pool, you can grep your kernel 
logs. On my physical PC with real hardware, interruptions etc. it's quite fast: 
5 seconds.

-- Logs begin at mar. 2016-01-26 07:54:37 CET, ...
...
juin 06 18:34:47 smithers kernel: random: systemd urandom read with 2 bits of 
entropy available
...
juin 06 18:34:52 smithers kernel: random: nonblocking pool is initialized

I get that the "kernel: random: systemd urandom read with 2 bits of entropy 
available" message comes from the kernel when systemd reads from /dev/urandom 
whereas the pool is not initialized yet.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Christian Heimes]

Christian Heimes added the comment:

Cory, thanks for summing it up for us. I totally agree with you. In my opinion 
it is troublesome to have different behavior on platforms. We can implement a 
workaround for Linux, but not for BSD. Or would O_NONBLOCK cause read() to fail 
with EWOULDBLOCK on /dev/urandom device?

It might be secure enough to use srandom() / random() instead of /dev/urandom 
in some platforms. It still won't do any good on platforms like Raspberry Pie 
since the SoC has no RTC. Without a RTC the clock is not set yet. It happens 
much later in the boot phase when network is available.

I don't see a cross-platform solution that is able to handle this super-special 
case without opening a potential security issue for the majority of users.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

So, in short, you don't know.

#25003 is about Solaris, and the reporter clearly had the expectation that 
/dev/urandom would never block.  The documentation on Linux is clear: 
/dev/urandom will never block.  That's two.

This "StackExchange" discussion:
  
http://security.stackexchange.com/questions/42952/how-can-i-measure-and-increase-entropy-on-mac-os-x
suggests that the Yarrow-based /dev/random and /dev/urandom on OS X will *both* 
degrade to PRNG if insufficient entropy is present.  Thus they are are *both* 
like /dev/urandom, and *neither* will ever block.

The salient quote is this, from the random(4) manpage on OS X:
"If the SecurityServer system daemon fails for any reason, output quality will 
suffer over time without any explicit indication from the random device itself."

That sure sounds like bad quality PRNG random bits to me.  So that's three.

Again: ISTM that the universal expectation is that /dev/urandom will never 
block.  Therefore os.urandom() should also never block.  That it blocks in 
3.5.0 and 3.5.1 is a performance regression and should be fixed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Cory Benfield]

Cory Benfield added the comment:

I have never seen it block in person, but I also wouldn't expect to. OS X's 
blocking guarantees are the same as FreeBSD's: that is, both /dev/random and 
/dev/urandom block until sufficient entropy has been gathered at startup and 
then never block again.

This means that, on OS X, in practice, /dev/urandom does never block, because 
you basically can't run user code early enough to encounter this problem.

FreeBSD again has the exact same behaviour: /dev/urandom is a symlink to 
/dev/random, and both will block at startup until sufficient entropy is 
gathered and never again. This is the bigger risk for Python, because if Linux 
people want to use Python in their init system it's not unreasonable for 
FreeBSD folks to want to do it too.

This is why I'm concerned about this "solution": while there's no question that 
adding getrandom() made the situation worse on Linux, it has drawn our 
attention to the fact that Python is relying on Linux-only semantics of 
/dev/urandom on all Unices. That's probably not a good plan.

(The above is all facts. Everything in these parentheticals is opinion. Please 
disregard as appropriate.

I agree with the OS X devs in that I believe their implementation *is* better 
than Linux's: sorry Ted! There is no reason to be concerned about using a good 
kernel CSPRNG once sufficient entropy has been gathered *once*. The CSPRNG 
essentially "stretches" the entropy out into a long sequence of numbers, much 
like a cipher like AES "stretches" the entropy in the key across the entire 
cipherstream. Talking about "running out" of entropy in one of these devices is 
weird to me: as a blog post linked earlier mentions, it's like talking about 
"running out of key" in an encryption algorithm.

It seems to me, then, that Linux's /dev/random is wrong in most situations 
(because it sometimes blocks), and /dev/urandom is wrong in some situations 
(because it'll run before it has enough entropy to properly seed the CSPRNG and 
it won't tell you that that is what has happened). On OS X, the best of both 
worlds occurs: you get no random numbers until sufficient entropy has been 
gathered to seed the CSPRNG, and then you get good random numbers from that 
point on.

Please note: I am not a trained cryptographer. However, trained cryptographers 
have agreed with this set of sentiments, so I think I'm on pretty good ground 
here.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

Are you certain that /dev/urandom will block on Mac OS X if sufficient entropy 
is not available?  The dismissive tone ("this choice and distinction is not 
necessary") suggests that *their* implementation is superior, and it could 
hardly be seen as superior if sometimes it blocks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Cory Benfield]

Cory Benfield added the comment:

> If you read #25003, it's clear that /dev/urandom is a well-known UNIX 
> facility with well-known, predictable behavior.  One behavior that I'll draw 
> particular attention to now: it will never block.  If the system is low on 
> entropy, /dev/urandom will produce lower-quality random bits.

That's not accurate.

/dev/urandom is a well-known UNIX facility, yes, but it does not have 
consistent behaviour across Unices. The behaviour you've described here Larry 
is a well-known *Linux* behaviour.

However, on other Unices the behaviour is different. Here, for example, is the 
relevant man page from Mac OS X ("man 4 random"):

 /dev/urandom is a compatibility nod to Linux. On Linux, /dev/urandom will
 produce lower quality output if the entropy pool drains, while
 /dev/random will prefer to block and wait for additional entropy to be
 collected.  With Yarrow, this choice and distinction is not necessary,
 and the two devices behave identically. You may use either.

Note the specific wording here: "the two devices behave identically". That is 
to say, on OS X both /dev/random and /dev/urandom are identical devices, and 
that includes the fact that both will in principle block if used without 
sufficient entropy.

OS X's implementation is a direct descendent of FreeBSD's, so the same caveats 
apply there, and in fact all the BSDs have this exact same behaviour.

So, again, I repeat my objection from above: if the concern is that starting 
Python must never block, then Python must *never* read from /dev/urandom on 
startup. Otherwise, Python *can* block on BSDs (OS X included in principle, 
though in practice I doubt Apple will use Python that early in boot).

At this point I literally no longer care whether os.urandom() is just a wrapper 
around /dev/urandom: we can look back on this in 10 years and see how we feel 
about the choices made by core dev at that time. But if we're arguing that this 
issue is about "Python must never block at startup", then we really have to 
acknowledge that /dev/urandom *can block* on some Unices, and so is entirely 
unacceptable for reading at startup.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

I've revised my thinking on the subject.

First, my previous statements stand: Python startup must not block.  "import 
random" must not block.

Now I'm thinking that "os.urandom()" must not block, too.  Here's my reasoning.

--

If you read #25003, it's clear that /dev/urandom is a well-known UNIX facility 
with well-known, predictable behavior.  One behavior that I'll draw particular 
attention to now: it will never block.  If the system is low on entropy, 
/dev/urandom will produce lower-quality random bits.

Again I repeat myself: this is the *expected* behavior.  It is so completely 
the expected behavior, that today's special celebrity guest on the issue, Mr. 
Theodore Ts'o himself, added /dev/random specifically so it would be permitted 
to block when the system was low on entropy.  He *did not change* the behavior 
of /dev/urandom.  He added a *new device*.

A well-informed engineer would see "os.urandom()" and predict (correctly) that 
Python has provided a thin layer over /dev/urandom.  Thus os.urandom() should 
provide the same well-known, predictable behavior as /dev/urandom.

It's fine to enhance os.urandom().  For example, it's fine to provide 
higher-quality bits where available.  It's fine to provide the function on 
Windows which doesn't have a /dev/urandom object.

What is *not* fine is to degrade its behavior.  /dev/urandom is known to never, 
ever block.  This is a *feature*.  os.urandom(), therefore, must also never, 
ever block.

Yes, this means that on these cloud instances with no entropy (yet), 
os.urandom() may return these low-quality random bits.  Just like /dev/urandom 
does.

If I understand the APIs correctly, I'm fine with os.urandom() calling 
getrandom(,,GRND_RANDOM|GRND_NONBLOCK).  If that fails with EAGAIN it should 
fall back to reading from /dev/urandom, or getrandom(,,GRND_NONBLOCK) if that 
makes sense.  (IDK if that's Linux-specific; if it is I suppose /dev/urandom is 
the more cross-platform way to go.)

--

If this is seen as the end of the world by the crypto guys in the thread, let 
me say that I'm willing to consider adding a new function in 3.5.2.  I would 
propose it be spelled "os.getrandom(n, block=True)".  Crypto code could use 
this function if available, and fall back to os.urandom() where it was not.  
This means you're covered: in 3.5.0 and 3.5.1 you use os.urandom(), and in 
3.5.2+ you use os.getrandom(), and in both circumstances you'll block if 
there's insufficient entropy.

--

p.s. Colm Buckley: you notice how dstufft's patch got a "review" link, and none 
of the patches you posted got one?  That's because his is based on the current 
3.5 repo and yours aren't.  This "review" link is very useful in reading your 
patches.  Please in the future try to base your patches against 3.5 trunk.  
It's easy:
% hg clone https://hg.python.org/cpython/ 
% cd cpython
% hg up -r 3.5
(do your work here)
% hg diff > patchfile
(upload patchfile)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Christian Heimes]

Christian Heimes added the comment:

Thanks Theodore!

Your analysis was exactly what I was aiming for when I requested a thorough 
security analysis in form of a PEP. The correct choice of CPRNG is important 
for the overall security. I'm mostly concerned with the behavior of 
os.urandom(), which is Python's preferred CPRNG for any critical data from 
session keys to key material. Several people including me have been pushing 
developers towards os.urandom() for many years. For that reason I put up a veto 
against any hasty change. Larry could my bluff.

The hash randomization for hashing of strings and bytes is explained in PEP 
456, https://www.python.org/dev/peps/pep-0456 . I wrote the PEP and added DJB's 
and JP Aumasson's SipHash24 as PRF. The 24 byte Py_HashSecret struct contains 
two keys for SipHash24 and another 8 byte key for randomization of expat XML 
library, https://hg.python.org/cpython/file/tip/Include/pyhash.h#l34 .

For short-running scripts early in the boot phase, hash randomization is not 
required at all. It is only relevant for applications that reads untrusted data 
from potentially malicious peers. GET dict of HTTP requests is a famous 
example. Hash randomization can already be disabled or set to a fixed value by 
using an env var. I argued to add an option that falls back to a different 
CPRNG and sets other options at the same time (#16499) but Larry (release 
manager of 3.5) is against any option. He wants Python to always start in a 
timely fashion without any extra arguments.

Your suggestion should fix the issue on Linux (GRND_NONBLOCK, fall back to 
srandom()), although I would rather use gettimeofday() with t.tv_sec + 
t.tv_usec. I'm still concerned how we should address the issue on BSD. As far 
as I am familiar with BSD, all reads from the Kernel's CPRNG are blocking until 
the CPRNG is seeded.

I can bring up the issue with Lennart, if it is really necessary (one advantage 
of working for Red Hat ;) ). I'm going to ping JP Aumasson to get his feedback.

--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Theodore Tso]

Theodore Tso added the comment:

I ran the experiment Colm asked me to run --- and yes, if you boot a system 
with Python 3.5.1 with the boot options "init=/usr/bin/python3", you're going 
to have a bad time.   The problem is that in a KVM environment where things are 
very quiet, especially if you are using a tickless kernel, if python calls 
getrandom(2), it will block since the entropy pool hasn't been initialized yet. 
  But since we aren't doing anything, the system becomes completely quiescent 
and so no entropy can be collected.  If systemd tries to run a python script 
very early in the boot process, and does this in a way where no further boot 
time activity takes place until the python script exits, you can indeed 
deadlock the system.

The solution is I think what Donald suggested in msg267746, which is to use 
GRND_NONBLOCK for initializing the hash which gets used for the dict, or 
whatever it's used for.   My understanding is that this is not a long-term 
cryptographic secret, and indeed it will be thrown away as soon as the python 
interpreter exits.  Since this is before networking has been brought up, the 
denial service attack or whatever requires that you use a strong SipHash for 
your Python dictionaries shouldn't be a problem.   (Which I gather has 
something to do with this?   
https://events.ccc.de/congress/2011/Fahrplan/attachments/2007_28C3_Effective_DoS_on_web_application_platforms.pdf)

Now, I can see people being concerned that if Python *always* initializes its 
hash dictionaries using getrandom with GRND_NONBLOCK, it might be opening up a 
DOS attack.   Well, in practice, once the boot sequence continues and the 
system is actually doing some real work, within a few seconds the random number 
generator will be initialized so in practice it won't be an issue once the 
system has booted.

If you want to be really paranoid, I suppose you could give some kind of 
command-line flag which tells Python to use GRND_NONBLOCK for the purposes of 
initializing its hash table for its dictionary, and only use it in the boot 
path.   In practice, I suspect very early in the systemd boot path, before it 
actually starts running the boot scripts in parallel, is the only place where 
you are likely going to run into this problem, so making it be a flag that only 
systemd scripts have to call is probably the right thing to do.   But I'll let 
someone else have the joys of negotiating with Lennart, and I won't blame the 
Python devs if using GRND_NONBLOCK unconditionally is less painful than having 
to work with the systemd folks.  :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: [issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[M.-A. Lemburg]

On 07.06.2016 22:27, Theodore Tso wrote:
> 
> Secondly, when I decided to add this behavior to getrandom(2), it was because 
> people were really worried that people would be using /dev/urandom for 
> security-critical things (e.g., initializing ssh host session keys, when 
> they'd _really_ rather not the NSA have be able to trivally pwn the server) 
> before it had been completely initialized.   (And if it is not completely 
> initialized, it would be trivially and embarassingly easy.  See 
> https://factorable.net/weakkeys12.extended.pdf for an example of where this 
> was rather disastrous.)

Thanks, Theodore, for this paper reference. It provides convincing
arguments that going back to the Python 3.4 behavior is indeed not
a good idea - even though I'm still not convinced that the main
use case for os.urandom() is cryptography. Most people will
simply use it to seed their Mersenne Twisters, like the random
module does too.

Now, raising an exception instead of blocking would likely cause
even more breakage, so I'm with Colm in keeping Victor's patch
and applying the fix to not block in dev_urandom_noraise().

We still need to fix the random module issue, though.

For 3.6, I wish we could have the getrandom() API exposed as
os.getrandom(), with all options available to applications.
That way, the application can decide what is best for them.

-- 
Marc-Andre Lemburg
eGenix.com

___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

Ted -

I'd suggest the following to test.

Boot an arbitrary Linux system with init=/usr/bin/python3 (assuming filesystems 
mounted etc). Python 3.5.1 (on Linux) will call getrandom() in blocking mode 
very early in its startup; if this happens before the pool is initialized, 
Python will fail to start. Given that ~nothing else will be happening, I'm 
interested to see what happens to the entropy pool, and whether getrandom() 
returns.

Haven't tried this myself, but it should work.

Colm

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Donald Stufft]

Donald Stufft added the comment:

Thanks for weighing in Theodore, I think that matches what Colm's last 
suggestion was, and what I was personally OK with. To seed our SipHash function 
using GRND_NONBLOCK since it's likely that will be fine, and worst case we're 
just using it for some hash tables.

Then for our os.urandom binding, we should use getrandom() without 
GRND_NONBLOCK since we don't know why someone is calling os.urandom, but we 
know in practice people are using it for cryptographic keys and the like.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Theodore Tso]

Theodore Tso added the comment:

Hi.   Colm alerted me to this bug, so I thought I would chime in as the author 
of Linux's getrandom(2) function.

First of all, if you are OK with reading from /dev/urandom, then you might as 
well use getrandom's GRND_NONBLOCK flag.  They are logically equivalent.

Secondly, when I decided to add this behavior to getrandom(2), it was because 
people were really worried that people would be using /dev/urandom for 
security-critical things (e.g., initializing ssh host session keys, when they'd 
_really_ rather not the NSA have be able to trivally pwn the server) before it 
had been completely initialized.   (And if it is not completely initialized, it 
would be trivially and embarassingly easy.  See 
https://factorable.net/weakkeys12.extended.pdf for an example of where this was 
rather disastrous.)

Why didn't I make /dev/urandom blocking?  Because a lot of people would whine 
and complain.   But getrandom(2) was a new interface, and so this was something 
I could do.   Now, before I decided to do this, I did do some benchmarks, and 
pre-systemd in practice on real hardware (e.g., x86 servers and laptops), I 
observed that you would actually see a message indicating that we had gathered 
128 bits of entropy long before the root file system had been mounted.With 
systemd, I observed that udevd was trying to read from /dev/urandom when we had 
only gathered an estimated 7 bits of entropy --- but I devoutly hoped that 
udevd wasn't doing anything super security critical, and trying to get the 
systemd people to change what they are doing is mostly like trying to teach a 
pig to sing, so I let it be.However, in practice within a single digit 
number of seconds, the kernel printk indicating that random driver had 
considered itself initialized came quickly enough that I figured it would 
 be safe to do.

If people are claiming that they are seeing cases where it takes over 90 
seconds for the random number generator to initialize itself, please contact me 
directly; I'd love to know more, because that's input I would very much like to 
have.

However, at the end of the day, on certain hardware, if you don't have a source 
of initial entropy because the system doesn't have enough real hardware with 
real sources of entropy --- or if you don't trust your friendly cloud provider 
to provide you with some entropy from the hypervisor's entropy pool via 
virtio-random --- you can either (a) decide to pretend you are secure, when you 
really aren't, (b) wait, or (c) decide that you don't *really* need a secure 
source of randomness because you're really just initializing a hash for some 
associative array, and in fact srandom(time(0)) would have been fine, and you 
were using getrandom(2) or /dev/urandom just because you wanted to feel like 
one of the cool kids.

That being said, I do know of one potential issue which is if you happening to 
be using Microsoft Azure, the way the virtualized interrupt works, we weren't 
actually getting any entropy, and this was something I didn't discover until 
someone sent me a patch.  I have a patch[1] queued up in the random.git tree 
for the next kernel merge window to address that issue for Microsoft Azure 
servers. 

[1] 
http://git.kernel.org/cgit/linux/kernel/git/tytso/random.git/commit/?h=dev=8748971b4f5e322236154981827bf43dec4dc470

On a Google Compute Engine (GCE) system, I just did a quick test, and the 
"random: non-blocking pool initialized" message appears 5.64 seconds after the 
system is booted.  The changes I have queued up in random.git should reduce 
that to under a second.

All of this is neither here nor there, though.  The big question is *what* does 
Python expect to do with the randomness.  If you are just using it for 
computational simulation, you can do whatever you want.   If you are using it 
to create long-lived secrets that are intended to be secure against the 
depredations of a Nation-State's intelligence service, and you are on a system 
which really has almost no entropy available to be collected, then falling back 
to reading from /dev/urandom or using GRND_NONBLOCOK is going to be the 
equivalent of saying La-La-La-La-La-Nobody-Knows-How-Secure-I-Am while keeping 
your ears plugged.(Now, if you are on an Intel system with RDRAND, and you 
trust Intel not to have given a back door to the NSA, you probably are safe, 
because we do actually mix in RDRAND.  On the other hand, if you are using some 
crappy ARM SOC for some Internet of Things device, and are firing up Python 
right after the system boots for the first time, and creating long-lived RS
 A private keys within milliseconds after the system is first booted --- please 
tell me so, I can avoid your product like the Plague.  :-)

--
nosy: +Theodore Tso

___
Python tracker 

___

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

I've spoken with Ted Ts'o (one advantage of working for Google) and taken a 
look in the Linux kernel source, and things are actually better than we'd 
feared.

Firstly, calling getrandom() with GRND_NONBLOCK and a buffer size of less than 
or equal to 32 bytes will always succeed (so, for the hash seed initialization 
at least, the EAGAIN logic is superfluous - it's still possibly needed for the 
general case and other operating systems, though).

Secondly, the quality of the getrandom data *before* the kernel PRNG is 
initialized is still pretty good - it's seeded from a combination of RDRAND, 
interrupt timing, several kernel parameters like uname -a, and RTC. Ted is 
confident that at least 24 bytes of real entropy will be present by a few 
seconds into boot time (due to interrupts etc), and that the predictability of 
the data will be very low.

Finally - note that any network-facing applications are *extremely* unlikely to 
encounter this issue, as they will be started well after networking and other 
good entropy sources have started. In particular, getrandom() will no longer 
block once fastinit has completed (on my system, this was less than one second 
after kernel load).

In other words, I think we are very safe to proceed with changeset 9de508dc4837 
+ the nonblocking_urandom_noraise.patch

Note that this solves the problem for *Linux* - if other operating systems do 
indeed have blocking /dev/urandom reads, this still needs to be addressed. I am 
not aware of any reports from non-Linux systems, though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Donald Stufft]

Donald Stufft added the comment:

> Once this has been done, it will never block again, and happily send you poor 
> random data if the entropy pool has been completely wiped of any entropy data 
> - without telling you.

This doesn't actually happen in real life, once urandom has been initialized 
you will never be able to get "poor random" out of it. You will get 
cryptographically secure random out of it always. *ACTUAL* Cryptographers 
pretty much universally agree on this statement. You can even use them for 
cryptographic keys, no matter how long it's been since your system booted as 
long as the urandom pool has had a chance to initialize.

> Or put differently: Where is the attack vector that blocking behavior of 
os.urandom() would help remedy ?

Someone attempting to use cryptographic random before the urandom pool has been 
sufficiently initialized to provide said random.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Christian Heimes]

Changes by Christian Heimes :


--
nosy:  -christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 07.06.2016 21:12, Cory Benfield wrote:
> 
>> Python hash randomization only happens once.  So it's not a matter of how 
>> early we try the attack, it's a matter of how early we seed Python hash 
>> randomization.
> 
> Sorry Larry, I was insufficiently clear (relying on context from earlier). I 
> totally agree that Python startup should not block. I'm saying that having 
> getrandom() called in "blocking mode" for os.urandom, random.SystemRandom, 
> and secrets is not a DoS vector.

I'm not sure I follow. A delay of 90 seconds on startup of a VM
or container can easily lead the supervisor style management tool
to think that something is wrong and issue a retry. Depending on
the configuration it'll then try this a couple of times and
give up.

Overall, I have a hard time following the arguments.

To be clear: getrandom() on Linux is just a wrapper with some
additional control around /dev/random and /dev/urandom.

http://lxr.free-electrons.com/source/drivers/char/random.c#L1601

Unlike /dev/urandom, getrandom() without flag GRND_NONBLOCK will
block, but only in the case where the entropy pool has not been
initialized yet. Once this has been done, it will never block
again, and happily send you poor random data if the entropy pool
has been completely wiped of any entropy data - without telling
you.

So now, you're all arguing: oh my, it's so insecure to use
data from /dev/urandom when the entropy pool is not initialized.
But you're not worried about os.urandom() happily sending you data
which is no longer based on any external entropy half an hour later:

http://lxr.free-electrons.com/source/drivers/char/random.c#L1458

This doesn't make sense. Either you're worried all the time,
or you're not :-)

The whole discussion is centering around whether to block
on an uninitialized entropy pool or not. This can only happen
during startup. By falling back to reading /dev/urandom
in case of an uninitialized pool, you are reading data from
a not fully initialized pool, but you still get random data.
That's really all that's needed for basic operations like
hash seeding or seeding the PRNG in the random module.
And it's limited to Python processes which are run very
early in the VM/container startup phase.

Note that "uninitialized" only means that the kernel entropy
pool has not yet reached an "entropy level" of 128 (whatever
that means):

http://lxr.free-electrons.com/source/drivers/char/random.c#L676

It does not mean that you're just reading a list of zeros.

So given all this information, why is it that you get so
tangled up in wanting os.urandom() to block during system
startup ?

Or put differently: Where is the attack vector that blocking
behavior of os.urandom() would help remedy ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

I fear I may be changing my mind a little bit.  However, I skipped 
breakfast--and now it's looking like a late lunch--so I simply have to step 
away for a while.  Expect me to post in about two hours when I get some 
calories down and finally make up my tiny mind.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Donald Stufft]

Donald Stufft added the comment:

Colm,

Great, then I think there's general agreement, we just need someone to review 
the nonblocking_urandom_noraise.patch (which my C is not strong enough to feel 
comfortable doing). That still leaves the `import random` issue, but I think we 
can reopen #25420 and figure out over there what the right answer is for that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Cory Benfield]

Cory Benfield added the comment:

> Python hash randomization only happens once.  So it's not a matter of how 
> early we try the attack, it's a matter of how early we seed Python hash 
> randomization.

Sorry Larry, I was insufficiently clear (relying on context from earlier). I 
totally agree that Python startup should not block. I'm saying that having 
getrandom() called in "blocking mode" for os.urandom, random.SystemRandom, and 
secrets is not a DoS vector.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

> This is only a DoS vector if you can hit the server so early in the boot 
> process that it doesn't have enough entropy.

Python hash randomization only happens once.  So it's not a matter of how early 
we try the attack, it's a matter of how early we seed Python hash randomization.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Cory Benfield]

Cory Benfield added the comment:

> So you are intentionally accepting a new vector for DoS attacks, and calling
this non-reduced security?

This is only a DoS vector if you can hit the server so early in the boot 
process that it doesn't have enough entropy. The *second* enough entropy has 
been gathered getrandom() will never block again.

In essence, then, the situation where it becomes possible to DoS a server is 
entirely outside an attackers control and extremely unlikely to ever actually 
occur in real life: you can only DoS the server if you can demand entropy 
before the system has gathered enough, and if the server has managed to *boot* 
by then then the alternative is that it is incapable of generating secure 
random numbers and shouldn't be running exposed against the web anyway.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

Donald -

Oh, that's interesting; thank you. I guess that system-crontab-generator is 
being called before that in the boot process.

The most common trigger case, I guess, will then be cloud containers and VMs 
which are spun up for single applications. I think Larry's comments are still 
very valid.

Colm

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Donald Stufft]

Donald Stufft added the comment:

> specifically, early in the boot process, although not I think just on the 
> first boot of a system; Debian at least by default does not attempt to 
> preserve its entropy pool across a reboot.)

Look at /etc/init.d/urandom in the initscripts package in Jessie 
(https://anonscm.debian.org/cgit/collab-maint/sysvinit.git/tree/debian/src/initscripts/etc/init.d/urandom).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

To clarify what the various patches do:

3.5.1 as released: os.urandom and hash secret initialization both attempt 
getrandom() in preference to reading /dev/urandom. Under certain circumstances, 
this will block, possibly indefinitely.

Changeset 9de508dc4837: both os.urandom and hash secret initialization call 
getrandom() in nonblocking mode, falling back to (possibly low-entropy) 
/dev/urandom should getrandom() block due to lack of entropy.

Changeset 9de508dc4837 + nonblocking_urandom_noraise.patch: hash secret 
initialization calls getrandom() in nonblocking mode (ie: will always succeed, 
although with a silent fallback to low-entropy data if called when the system 
has no entropy). os.urandom will always block until there's enough entropy.

I think this final case implements what you need for the 3.5.2 RC.

The issue of "import random" still needs to be resolved; maybe we should 
de-merge #25420 and pursue Donald's approach there.

Thanks,

Colm

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Matthias Klose]

Matthias Klose added the comment:

On 07.06.2016 16:47, Christian Heimes wrote:
> 
> Christian Heimes added the comment:
> 
> PSRT VETO!
> 
> In the light of the upcoming release 3.5.2 I'm now putting on my PSRT hat 
> (Python Security Response Team) and proclaim a veto against any and all 
> changes to os.urandom(). The security properties of os.urandom() must not be 
> modified or reduced compared to 3.5.1. Please restore the behavior of 
> os.urandom().

So you are intentionally accepting a new vector for DoS attacks, and calling
this non-reduced security?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

That reminds me.  I want to be clear: I think it's preferable that os.urandom() 
blocks when insufficient entropy is available.  If Victor's patch changed that, 
it should be backed out.

(Since non-blocking urandom is useful, perhaps in 3.6 os.urandom() should take 
a new "block=True" parameter.  But it's too late to add it for 3.5.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

Larry -

To the first point:

The combination of Victor's changeset 9de508dc4837 (based on my patch) and my 
most recent nonblocking_urandom_noraise patch (which is on top of 9de508dc4837) 
will do what you suggest for the hash secret initialization - ie: it is allowed 
to fall back to predictable sources when there is insufficient entropy to 
securely seed it.

I suspect that it is simply impossible to reconcile "os.urandom will never 
block" with "os.urandom is always cryptographically reasonable". If the system 
has no entropy, it has no entropy. The only escape I see is to add an exception 
condition, instead of the silent fallback which some platforms currently have. 
There is a judgement call to be made here; whether silent fallback is 
acceptable or not.

As Donald points out, this will fail only in very unusual circumstances 
(specifically, early in the boot process, although not I think just on the 
first boot of a system; Debian at least by default does not attempt to preserve 
its entropy pool across a reboot.)

This should not affect things like web servers etc. as they start much later in 
the boot process; in particular after networking has started, which I believe 
is the principal source of entropy for /dev/urandom.

Colm

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Donald Stufft]

Donald Stufft added the comment:

> As a final note, let me steer you towards this comment in Python/random.c:
>
> /* Issue #25003: Don' use getentropy() on Solaris (available since
>  * Solaris 11.3), it is blocking whereas os.urandom() should not block. */
>
> Yes: we already had this discussion for Solaris, nine months ago, on issue 
> #25003.  Both Guido and Tim Peters were involved in the discussion.  The 
> decision there: use lower-quality random bits to seed the MT when importing 
> the random module.  Keeping the slowdown was so obviously wrong it wasn't 
> even debated.

I will point out, that was a somewhat different situation as ``getentropy`` on 
Solaris is more like /dev/random in that it tries to decide how much 
random-ness is in the pool and will randomly block throughout the execution of 
the program. The ``getrandom()`` call on Linux (and Solaris) will, by default, 
only block on the first boot at the very beginning before the kernel has 
collected enough entropy.

I don't think this changes anything, I just want to be clear because there are 
two kinds of "blocking" in this discussion, one that only occurs in very 
specific scenarios and one that occurs regularly in the operation of the 
program.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

Everybody: let's drop discussing "hashlib" unless someone says it actually is a 
problem.  I think it was always, as we say in English, a "red herring".


> The secret for SipHash is composed of two 64bit integers. The entire 
> _Py_HashSecret_t struct is 24 bytes. The remaining 8 bytes are used for XML 
> hash randomization of libexpat. Only the manual seed with PYTHONHASHSEED is a 
> 32bit integer which is stretched to 24 bytes with a LCG.

Okay, I have misunderstood the code.  Have I misunderstood the strength of 
SipHash?  Is it regarded as "cryptographically secure"?

Predictability of the hash function on web servers was the original use case of 
the "hash seed"; I remember a demonstration of an attack where the attacker 
produced pathologically bad hash behavior on a Python-based web server with 
very little data.  So it seems like web servers running on cloud instances is 
exactly the sort of use case where we'd want less-predictable hashing.

--

Nevertheless, a 90 second startup time is simply unacceptable.  I am officially 
making a pronouncement as Release Manager: Python 3.5 *must not* take 90 
seconds to start up under *any* circumstances.  I view this as a performance 
regression, and it is and will remain a release blocker for 3.5.2.

Python *must not* require special command-line flags to avoid a 90 second 
startup time.  Python *must not* require a special environment-variable to 
avoid a 90 second startup time.  This is no longer open to debate, and I will 
only be overruled by Guido.

--

If I understand the technical issues correctly, here's how I expect it to work. 
 For seeding the hash randomization, and seeding the _inst in the random 
module, we will use getrandom() in a non-blocking way (GRND_NONBLOCK?).  If it 
succeeds, we use those bits.  If it fails because it would have blocked 
(EAGAIN?), we fall back to a less-random source of random bits.  Under no 
circumstances will Python block when seeding the hash randomization function or 
seeding the MT for the random module.

This means cloud instances may inadvertently use lower-quality hash 
randomization seeds.  I judge this as obviously better than cloud instances 
taking 90 seconds to start up.  Also, as Christian points out, the people 
running these cloud instances should be managing their entropy pools anyway.  
Additionally, there are many uses of cloud instances that aren't exposed to 
tainted data that permit these predictable-hash abuses.

--

As a final note, let me steer you towards this comment in Python/random.c:

/* Issue #25003: Don' use getentropy() on Solaris (available since
 * Solaris 11.3), it is blocking whereas os.urandom() should not block. */

Yes: we already had this discussion for Solaris, nine months ago, on issue 
#25003.  Both Guido and Tim Peters were involved in the discussion.  The 
decision there: use lower-quality random bits to seed the MT when importing the 
random module.  Keeping the slowdown was so obviously wrong it wasn't even 
debated.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Donald Stufft]

Donald Stufft added the comment:

I've attached a minimal patch for making it so ``import random`` does not 
block, it does this by changing what the default instance of Random() is seeded 
with, from os.urandom() to the time based fallback it currently employs. It 
does not change the behavior of any documented behavior that I can see (it's 
documented that calling seed(None) or seed() will use urandom if available).

This could be improved by mixing in id(self) and using SipHash or LCG on the 
value, but this represents a minimal patch that is already possible in cases 
where os.urandom doesn't exist.

--
Added file: http://bugs.python.org/file43282/no-urandom-by-default.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Christian Heimes]

Christian Heimes added the comment:

On 2016-06-07 19:36, Donald Stufft wrote:
> 
> Donald Stufft added the comment:
> 
>> Possible resolutions:
>>  * accept possible low-entropy initialization of the hash secret; using the 
>> patches supplied here by myself and Victor.
>>  * add a command-line flag to disable "strong" initialization of the hash 
>> secret (or revive the old -R flag).
>>  * simply require user-space workarounds like setting PYTHONHASHSEED
> 
> I think either the first or second here are good solutions, the third is kind 
> of crummy on it's own because it's not always possible to pass in an 
> environment variable. Pairing the third with a CLI flag option might work out 
> nice though, perhaps a -XPYTHONHASHSEED=(random/int()) or something. Then 
> folks who are in early boot can easily just hardcode a hash seed, removing 
> the need to hit the entropy pools while still maintaining strong random for 
> everyone else.
> 
> So I guess I would lean towards adding a CLI flag, but just allowing SipHash 
> to fall back to possibly bad randomness for it's initialization is OK.

I don't like the fact that applications can fall back to insecure RNG
without user involvement or warning.

Therefore I'm in favor of a command line argument that allows pyhash.c
to fall back to a less secure RNG. System scripts must use the -I option
(isolated mode without user-site dir and PY* env vars) anyway. The new
option would enable less secure RNG as fallback and -I.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Christian Heimes]

Christian Heimes added the comment:

On 2016-06-07 19:46, Larry Hastings wrote:
> 
> Larry Hastings added the comment:
> 
> Thank you for summarizing the debate. It made it a lot easier to 
> 
>> * blocking initialization of the hash secret. This occurs regardless of 
>> script contents; at present Python simply can't be used at all in 
>> low-entropy situations. I feel that this issue is a release blocker.
>>
>> Possible resolutions:
>>   * accept possible low-entropy initialization of the hash secret; using the 
>> patches supplied here by myself and Victor.
>>   * add a command-line flag to disable "strong" initialization of the hash 
>> secret (or revive the old -R flag).
>>   * simply require user-space workarounds like setting PYTHONHASHSEED
> 
> The latter two approaches are unacceptable IMO.  They result in a poor user 
> experience.  Python should do the "right" thing by default; the "right" thing 
> includes not taking 90 seconds to start up.
> 
> By process of elimination, this leaves only the first approach as viable.  
> Ergo, let's do that.
> 
> The hash secret is a 32-bit integer, even on 64-bit builds of Python.  It is 
> not and cannot be cryptographically secure.  It's frankly ridiculous to fret 
> about "strong" initialization of it at the cost of a 90 second startup time.
> 
> (For posterity: when people mention "SipHash", they're talking about the 
> hashing algorithm used for str/dict/etc.  The seed for SipHash is the "hash 
> secret" we're talking about here.)

The secret for SipHash is composed of two 64bit integers. The entire
_Py_HashSecret_t struct is 24 bytes. The remaining 8 bytes are used for
XML hash randomization of libexpat. Only the manual seed with
PYTHONHASHSEED is a 32bit integer which is stretched to 24 bytes with a LCG.

Christian

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Donald Stufft]

Donald Stufft added the comment:

> I don't follow whose patch does what.  But here's what I find acceptable, 
> from a high level.
> 
> * The semantics as presented by the documentation must be preserved.  
> os.urandom() and other operations that declare they're safe for cryptographic 
> use must remain safe for cryptographic use.
> * "import random" must not block.
> * "import hashlib" must not block.
>
> Is there a patch set that accomplishes that?

I *think* nonblocking_urandom_noraise.patch will solve the 90+ second start up 
without affecting os.urandom which should solve the first one (once the already 
applied patch gets reverted), but I'm afraid I don't know C well enough to 
meaningfully review that for accuracy.

None of the current patches solve the second without invalidating the first, 
but it would be, I believe, an additional patch ontop of 
nonblocking_urandom_noraise.patch.

The third is already the case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

Thank you for summarizing the debate. It made it a lot easier to 

> * blocking initialization of the hash secret. This occurs regardless of 
> script contents; at present Python simply can't be used at all in low-entropy 
> situations. I feel that this issue is a release blocker.
>
> Possible resolutions:
>   * accept possible low-entropy initialization of the hash secret; using the 
> patches supplied here by myself and Victor.
>   * add a command-line flag to disable "strong" initialization of the hash 
> secret (or revive the old -R flag).
>   * simply require user-space workarounds like setting PYTHONHASHSEED

The latter two approaches are unacceptable IMO.  They result in a poor user 
experience.  Python should do the "right" thing by default; the "right" thing 
includes not taking 90 seconds to start up.

By process of elimination, this leaves only the first approach as viable.  
Ergo, let's do that.

The hash secret is a 32-bit integer, even on 64-bit builds of Python.  It is 
not and cannot be cryptographically secure.  It's frankly ridiculous to fret 
about "strong" initialization of it at the cost of a 90 second startup time.

(For posterity: when people mention "SipHash", they're talking about the 
hashing algorithm used for str/dict/etc.  The seed for SipHash is the "hash 
secret" we're talking about here.)


> * blocking random reads during import hashlib or import random. This is more 
> complex, as we need to take developer intentions into account. I do *not* 
> think that these are release blockers as there are reasonably easy 
> workarounds, however the fact remains that there has been a regression in 
> Python's behavior on Linux.
> 
> Possible resolutions:
> 
>   * accept Victor's existing changeset without my nonblocking_urandom_noraise 
> patch, which makes _PyOS_URandom nonblocking in all Linux cases.
>   * resolve as above (both Victor's and my patches), and require that 
> applications be modified to work correctly
>   * require modifications to hashlib.py and random.py to use nonblocking 
> sources and/or raise exceptions accordingly.

I don't follow whose patch does what.  But here's what I find acceptable, from 
a high level.

* The semantics as presented by the documentation must be preserved.  
os.urandom() and other operations that declare they're safe for cryptographic 
use must remain safe for cryptographic use.
* "import random" must not block.
* "import hashlib" must not block.

Is there a patch set that accomplishes that?

--

If this means that random.random() may be seeded with poor-quality random bits, 
so be it.  As I think I already stated in this thread: there are many 
non-cryptographic uses for random.random().  And the documentation for the 
random module already states that it's not suitable for cryptography.  So 
making it block in order to procure a cryptographically-strong seed is 
counterproductive.

Also, I think the constraint "import hashlib must not block" is a non-issue.  I 
preserved it above just in case I missed something.  But Colm was the one who 
suggested "import hashlib" was blocking, and he quickly said afterwards that he 
was mistaken.  In any case a quick review of the code suggests that hashlib 
never uses getrandom and thus should not currently block.  Unless someone says 
otherwise, I'll assume hashlib is fine, never blocks on import, and thus 
requires no modification.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Donald Stufft]

Donald Stufft added the comment:

> Possible resolutions:
>  * accept possible low-entropy initialization of the hash secret; using the 
> patches supplied here by myself and Victor.
>  * add a command-line flag to disable "strong" initialization of the hash 
> secret (or revive the old -R flag).
>  * simply require user-space workarounds like setting PYTHONHASHSEED

I think either the first or second here are good solutions, the third is kind 
of crummy on it's own because it's not always possible to pass in an 
environment variable. Pairing the third with a CLI flag option might work out 
nice though, perhaps a -XPYTHONHASHSEED=(random/int()) or something. Then folks 
who are in early boot can easily just hardcode a hash seed, removing the need 
to hit the entropy pools while still maintaining strong random for everyone 
else.

So I guess I would lean towards adding a CLI flag, but just allowing SipHash to 
fall back to possibly bad randomness for it's initialization is OK.

>  * accept Victor's existing changeset without my nonblocking_urandom_noraise 
> patch, which makes _PyOS_URandom nonblocking in all Linux cases.
>  * resolve as above (both Victor's and my patches), and require that 
> applications be modified to work correctly
>  * require modifications to hashlib.py and random.py to use nonblocking 
> sources and/or raise exceptions accordingly.

Of these, I think random.py should just not use a CSPRNG, it's not required for 
it so there's no reason to do it. I don't think there's actually any problem 
with hashlib, I don't see any use of random in it.


> I would be interested in PSRT's assessment of an actual attack on a 
> predictable hash secret 

For something like systemd-crontab-generator, basically nothing-- for anything 
short lived or which does not provide a means for arbitrary users to put data 
into a dictionary. IOW, it's largely persistent network services.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

Larry -

I see at least two issues here, although they are related:

* blocking initialization of the hash secret. This occurs regardless of script 
contents; at present Python simply can't be used at all in low-entropy 
situations. I feel that this issue is a release blocker.

Possible resolutions:
  * accept possible low-entropy initialization of the hash secret; using the 
patches supplied here by myself and Victor.
  * add a command-line flag to disable "strong" initialization of the hash 
secret (or revive the old -R flag).
  * simply require user-space workarounds like setting PYTHONHASHSEED


* blocking random reads during import hashlib or import random. This is more 
complex, as we need to take developer intentions into account. I do *not* think 
that these are release blockers as there are reasonably easy workarounds, 
however the fact remains that there has been a regression in Python's behavior 
on Linux.

Possible resolutions:

  * accept Victor's existing changeset without my nonblocking_urandom_noraise 
patch, which makes _PyOS_URandom nonblocking in all Linux cases.
  * resolve as above (both Victor's and my patches), and require that 
applications be modified to work correctly
  * require modifications to hashlib.py and random.py to use nonblocking 
sources and/or raise exceptions accordingly.

I see these largely as policy decisions rather than technical ones. The 
security implications of the first issue are fairly small (I would be 
interested in PSRT's assessment of an actual attack on a predictable hash 
secret); of the second issue rather larger and probably unquantifiable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

Donald -

To be clear - no import of random or of hashlib is required to trigger this 
issue. The null script alone triggers the issue; the Python hash secret is 
initialized at startup regardless of script contents.

Yes, there is a race condition at system boot which we can probably resolve 
with userspace manipulations. I still feel that having Python hang indefinitely 
under certain circumstances, even when the application does not require any 
entropy, is a violation of the principle of least surprise. At the very least, 
there should be a command-line flag to disable "secure" initialization of the 
hash secret.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Donald Stufft]

Donald Stufft added the comment:

>  ISTM that the happy middle ground would be:
> * seed the random module with non-cryptographically-secure random bits
> * lazily seed hashlib

I don't think it was actually hashlib that was causing the problem, but rather 
the initialization of SipHash, it just so happened that hashlib was the first 
import and thus was getting the blame. I could be wrong about that though. In 
any case, I think it's perfectly reasoanble to seed the random module with 
non-cryptographically-secure random bits and if applicable lazily seed hashlib.

I also think it's reasonable to seed SipHash with possibly 
non-cryptographically-secure random bits since it's likely to only be a problem 
early on in the boot cycle. Another option is as Colm suggestion, allowing the 
inverse of the old -R flag, to turn off hash seed randomization from the CLI so 
that scripts that run early on in the boot process can disable hash seed 
randomization and not require reading from urandom (assuming of course, they 
don't do something that explicitly calls os.urandom).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Donald Stufft]

Donald Stufft added the comment:

> Please note: this is *not* just a VM/cloud issue. This is observed on 
> physical standalone systems.

But it should only occur on initial boot I believe? AFAIK all of the major 
linux vendors have stored a seed file once the machine has been booted and the 
pool has been initialized to use to seed the pool on subsequent boots. I 
suppose it's possible that 
/lib/systemd/system-generators/systemd-crontab-generator is running prior to 
Debian reseeding the pool from that seed file, but it seems like it should be 
doing the reseeding as early as possible?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Larry Hastings]

Larry Hastings added the comment:

> PSRT VETO!

This is an amusing concept, but membership in the PSRT does not empower you 
with a "veto".

On the other hand, being Release Manager does give me some say here.


>  You wouldn't add a workaround for broken CPU instructions to math.c or 
> semi-functional network card to socket.c, would you?

Well, yes, of course we would, if we had to.  Consider the F00F bug.  Happily 
the operating systems handled that one for us.

It is unreasonable for Python startup to take 90 seconds, poorly-configured 
cloud virtual machine or otherwise.  And there are many, many uses of the 
random module and hashlib that don't require CPRNG.

On the other hand, people who need cryptographic-strength random bits should be 
able to get them.  And the documentation literally does state that os.urandom() 
is a source of cryptographically-suitable random bytes.

ISTM that the happy middle ground would be:
 * seed the random module with non-cryptographically-secure random bits
 * lazily seed hashlib

Am I missing something, besides the anxiety of making this sort of change four 
days before I tag 3.5.2 RC2?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

... the script is /lib/systemd/system-generators/systemd-crontab-generator, 
although that's not hugely germane to the discussion. Arranging for 
PYTHONHASHSEED to be set while it's called wouldn't be impossible of course, 
although a command-line flag would be easier and probably safer.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

Christian -

Please note: this is *not* just a VM/cloud issue. This is observed on physical 
standalone systems.

The issue (on Debian) is that the Python script  is called very early in 
the boot process; in particular before most hardware initialization is done. As 
there are yet no network or USB devices configured, there is no entropy pool to 
speak of. We observe that getrandom() blocks apparently indefinitely under 
these circumstances (even though this script has no requirement for random data 
apart from the hash secret).

My final suggestion is that we return to using a command-line flag to indicate 
our preferences regarding hash seed initialization; although reverse the sense 
compared to the -R flag in 3.2.3 (ie: default is to use strong initialization, 
but allow the user to over-ride just as though PYTHONHASHSEED were set in the 
environment.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 07.06.2016 16:59, Christian Heimes wrote:
> 
> Christian Heimes added the comment:
> 
> On 2016-06-07 16:49, Marc-Andre Lemburg wrote:
>> This gives people a clear choice and doesn't cause people
>> to have to reconsider using the random module or wait for
>> Python hash randomization to initialize itself when using
>> Python during VM/container/system startup.
>>
>> I don't really appreciate this approach to break Python in
>> cloud setups just because some entropy pool is not initialized,
>> which only a tiny fraction of users care about. It doesn't
>> make Python land a better place.
> 
> VM and cloud setup without a proper CPRNG source are plain broken. True
> fact!
> 
> Secure entropy sources are a fundamental resource for all modern
> applications. Please start treating CPRNG like RAM, CPU or disks. You
> wouldn't add a workaround for broken CPU instructions to math.c or
> semi-functional network card to socket.c, would you?

For security relevant applications, I agree and for those
I question the use of os.urandom() altogether (see my other
replies), but for everything else, a PRNG is just fine.

I'm repeating myself, but making users believe that an
entropy source is more important that preventing a
denial of service just won't work out.

You're position is quite similar to the one that others
have taken with the I/O encoding in Python3. Their stance
was "fix your system and it'll work". Well, tell that
to 9-year olds who want to learn Python.

Likewise, it may be easy for you to track down the reason
why Python 3.5.1 isn't working in your VM or container,
but not everyone knows that there's an entropy source
which needs to be connected to your VM or container to
make things work - and even if they do know about the problem
they may not have the means to do so.

It's pretty much the same situation and the reason why
we have "practicality beats purity".

A hanging Python process is the worst of all user
experiences.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Christian Heimes]

Christian Heimes added the comment:

On 2016-06-07 16:49, Marc-Andre Lemburg wrote:
> This gives people a clear choice and doesn't cause people
> to have to reconsider using the random module or wait for
> Python hash randomization to initialize itself when using
> Python during VM/container/system startup.
> 
> I don't really appreciate this approach to break Python in
> cloud setups just because some entropy pool is not initialized,
> which only a tiny fraction of users care about. It doesn't
> make Python land a better place.

VM and cloud setup without a proper CPRNG source are plain broken. True
fact!

Secure entropy sources are a fundamental resource for all modern
applications. Please start treating CPRNG like RAM, CPU or disks. You
wouldn't add a workaround for broken CPU instructions to math.c or
semi-functional network card to socket.c, would you?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

Christian -

I would like to make one further comment:

The only reason getrandom() was used instead of /dev/random was to avoid 
wasting a file descriptor. The previous behavior was in use for many years with 
no security issues; it was changed for FD conservation reasons, not security 
reasons.

The change between 3.5 and 3.5.1 caused a very notable regression; the 
initialization of the hash secret can block indefinitely under circumstances 
which unfortunately are fairly common.

Persisting with the 3.5.1 behavior, in my opinion, violates the principle of 
least surprise - Python blocks at startup waiting for random data even when 
none is actually required by the application. The fallback to 3.5 behavior is 
only invoked under the single case where the system PRNG is uninitialized.

You are within your rights to request the reversion; however I want to point 
out again that the implications are the introduction of multi-minute delays 
into the startup times of hundreds of millions of systems, due to a change in 
*Python's* behavior.

Colm

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 07.06.2016 16:18, Donald Stufft wrote:
> 
> Donald Stufft added the comment:
> 
>> I just don't understand why you are apparently not willing to even consider 
>> compromises.
> 
> I have one thing that I hold immutable here, That os.urandom should use the 
> best interfaces provided by the OS to ensure that it always returns 
> cryptographically random data.
> 
> I don't care if SipHash is allowed to use lesser data.
> I don't care if os.urandom raises an exception or if it blocks if enough 
> entropy isn't available.
> I don't care if people are given an option that will let them maybe get bad 
> data (but will only work on Linux or older *nixes).
> 
> All I care about is that the default behavior of os.urandom gets data that is 
> generated using the best practices for that system, because that's what 
> people have been told to use for years and years.

Fine, but that's just your personal desire.

It's not something that we have documented anywhere - all these
years, we've told people that os.urandom() is an interface to
/dev/urandom and this was now changed in an incompatible way
without giving notice to anyone.

Why not expose os.getrandom() and then have folks that care
about having it block in the case of an uninitialized
entropy buffer use that ?

This gives people a clear choice and doesn't cause people
to have to reconsider using the random module or wait for
Python hash randomization to initialize itself when using
Python during VM/container/system startup.

I don't really appreciate this approach to break Python in
cloud setups just because some entropy pool is not initialized,
which only a tiny fraction of users care about. It doesn't
make Python land a better place.

This is similar to the approach that Python3 should always
determine the I/O encoding from the environment, without providing
sane defaults to even startup the interactive console or run a
pipe. It wasn't helping with the adoption of Python. We've
resolved that by making useful assumptions. We should do
the same here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Christian Heimes]

Christian Heimes added the comment:

PSRT VETO!

This ticket is turning into a bike-shedding discussion. In the light of the 
upcoming release 3.5.2 I'm now putting on my PSRT hat (Python Security Response 
Team) and proclaim a veto against any and all changes to os.urandom(). The 
security properties of os.urandom() must not be modified or reduced compared to 
3.5.1. Please restore the behavior of os.urandom().

Reasoning:
The security of our general audience is much more important than this special 
case. I agree that the problem of Python blocking in an early boot phase should 
be fixed. But under no circumstances must the fix affect security. For now 
please work around the issue with PYTHONHASHSEED or forwarding the host's 
entropy source into your virtualization environment.

Any change to os.urandom(), _Py_HashSecret (I'm the author of PEP 456) and 
Mersenne-Twister initialization of random.random() shall go through a formal 
PEP process. I'm willing to participate in writing the PEP. A formal PEP also 
enables us to ask trained security experts for review.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

Victor -

I see three options for 3.5.2:

* continue with the 3.5.1 behaviour, which blocks all python invocations in 
low-entropy situations. I think this is highly undesireable.

* apply my patches, which fixes the hash secret initialization but not 'import 
random'. This at least allows current Debian testing-track systems to boot 
properly ;)

* attempt to find a solution for #25420 which also addresses this issue. The 
original patch we submitted fixed both, but has encountered community 
objections from Donald and others.

The situation we're encountering is that it is *not possible* to use a sound 
PRNG under certain circumstances - if the system doesn't have entropy, it 
doesn't have entropy and there's not a lot to be done about it apart from wait.

I posit that an application which uses the random module has higher 
expectations of unpredictability, and therefore should take userspace measures 
to ensure entropy availability (as you suggest in msg253163 for example).

Note that the previous behavior (reading /dev/urandom) returns potentially 
unsafe data (as Donald and others point out). The only resolution to me seems 
to be modifying the behavior of the random module so that the buffer is 
initialized lazily (at first use, rather than at module import). This should be 
relatively straightforward, but I haven't had time to unpick all the logic of 
random.py to determine The Right Thing. Maybe Raymond can take a look at this?

In summary: I propose that the fix for this issue be implemented using the 
patches already discussed in this thread, and the fix for #25420 be implemented 
by modifying random.py.

Is this acceptable to everyone?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

The attached patch (against tip) is a very quick attempt to implement the 
desired behavior:

* add a "nonblocking" argument to py_getrandom()
* dev_urandom_python() sets this to 0, to request blocking operation
* dev_urandom_noraise() sets this to 1 to request non-blocking

As far as I can tell, dev_urandom_noraise() is only called by _PyRandom_Init() 
so only the hash secret initialization will be affected.

Victor; can you take a look and let me know what you think? This probably needs 
some comments etc. before pushing. I'm also a little unsure about the rest of 
the logic in dev_urandom_noraise - it should probably also be modified to not 
block in the event that urandom is unavailable.

--
Added file: http://bugs.python.org/file43278/nonblocking_urandom_noraise.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Stefan Krah]

Stefan Krah added the comment:

man urandom:

"A read from the /dev/urandom device will not block waiting for more entropy.  
As a result, if there is not sufficient entropy in  the
   entropy  pool,  the  returned  values  are  theoretically  vulnerable to 
a cryptographic attack on the algorithms used by the driver.
   Knowledge of how to do this is not available in the current unclassified 
literature, but it is theoretically possible  that  such  an
   attack may exist.  If this is a concern in your application, use 
/dev/random instead."


There was never any guarantee on Linux. Python is a language and not an 
application. Security checks should be done by applications or better during 
the OS startup.  Any properly configured Linux server will not have a problem, 
but it is not up to a language implementation to check for that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Donald Stufft]

Donald Stufft added the comment:

> I just don't understand why you are apparently not willing to even consider 
> compromises.

I have one thing that I hold immutable here, That os.urandom should use the 
best interfaces provided by the OS to ensure that it always returns 
cryptographically random data.

I don't care if SipHash is allowed to use lesser data.
I don't care if os.urandom raises an exception or if it blocks if enough 
entropy isn't available.
I don't care if people are given an option that will let them maybe get bad 
data (but will only work on Linux or older *nixes).

All I care about is that the default behavior of os.urandom gets data that is 
generated using the best practices for that system, because that's what people 
have been told to use for years and years.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[STINNER Victor]

STINNER Victor added the comment:

> nonblocking_urandom_noraise.patch doesn't fix the issue #21470.

Sorry, it's the issue #25420: "import random" blocks on entropy collection on 
Linux with low entropy

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[STINNER Victor]

STINNER Victor added the comment:

nonblocking_urandom_noraise.patch doesn't fix the issue #21470.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Christian Heimes]

Christian Heimes added the comment:

I'm with Donald here. Python must not reduce security just for a special case. 
It doesn't mean that we should not address and fix this special case -- just 
treat it as special.

1) For your use case, the hash randomization key for the SipHash PRN doesn't 
need to be 4 or 8 bytes of CPRNG. Since you are not dealing with lots of 
untrusted input from a malicious remote source, any unpredictable or even 
predictable value will do.

2) Your use case might be special enough to use a special build of Python. Too 
bad https://www.python.org/dev/peps/pep-0432/ is not ready yet. 

3) #21470 causes 'import random' to read os.urandom(2500) in order to 
initialize the MT state of random.random. I really don't understand why MT 
needs 2500 bytes of distinct CPRNG data. The module should rather read less 
data and then stretch it into a larger init vector. We could use SipHash for 
the job. In fact why does the MT use a CPRNG at all? It's not designed as CPRNG 
source and could be initialized from other sources (id(self), time()...) 
instead.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Donald Stufft]

Donald Stufft added the comment:

> I have no objection to *deliberate* invocations of the system RNG blocking if 
> needed. Presumably this behavior can be codified into the various APIs.
>
> My objection is *entirely* to _PyRandom_Init() calling a potentially-blocking 
> RNG source, before script parsing even begins.

It sounds like we might (somehwat?) be in violent agreement then.

If someone calls os.urandom() (or calls something that causes it to be called, 
e.g. secrets.py, random.SystemRandom, etc) then they should not get randomness 
from an un-initialized /dev/urandom by default. I have a preference for 
blocking until randomness is available, but an exception would be OK too.

I have no problem with the interpreter start up not blocking on entropy because 
no user invoked code caused that, and the security properties of SipHash that 
need really good random only really matter for long lived processes that 
processes a lot of user input-- IOW stuff that's unlikely to be started prior 
to the pool being initialized.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 07.06.2016 15:21, Donald Stufft wrote:
> 
> The patch causes a regression because I'm relying on the 3.5 behavior of 
> getting secure randomness from ``os.urandom`` via the ``getrandom()`` system 
> call (behavior that was documented in the Whats New in 3.5). The 3.5 behavior 
> also makes ``os.urandom`` behave the same on Windows, FreeBSD, OpenBSD, etc, 
> basically every major OS except for Linux.

The contention is not about using getrandom() to fetch data,
but the newly introduced and unwanted blocking nature during
system startup.

This was not documented anywhere and it's
a regression that causes major problems with using Python 3.5
on containers and VMs systems where startup time is of
essence.

You get the same blocking behavior when importing the random
module (see http://bugs.python.org/issue25420), even though it's
just seeding the global PRNG.

All these instances assume that os.urandom() does *not* block
and rightly so, since at the time they were written, os.urandom()
was a direct interface to /dev/urandom, which is documented
and designed to not block on Linux.

> And yes, it's not unusual for "bugs" to not be fixed if there is contention 
> about whether or not they are bugs at all and if they should be fixed. The 
> typical resolution path to not change anything unless there's broad 
> agreement, if that can't happen on bugs.p.o then escalate to python-dev, and 
> if it can't happen there then escalate to a PEP for a BDFL pronouncement.

Sure, we can have the discussion again on python-dev. I just
don't understand why you are apparently not willing to even
consider compromises.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

I have no objection to *deliberate* invocations of the system RNG blocking if 
needed. Presumably this behavior can be codified into the various APIs.

My objection is *entirely* to _PyRandom_Init() calling a potentially-blocking 
RNG source, before script parsing even begins. This basically prohibits Python 
from starting on systems where the system RNG is blocking. Linux is the only 
affected system *now* because the systemd crontab generator is the only Python 
script called before the RNG has initialized. Exactly the same issue would 
apply to any of the BSDs or Solaris, if /dev/urandom blocks as you describe. I 
want to be clear - this is not a Linux-specific issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Donald Stufft]

Donald Stufft added the comment:

> With the greatest respect, you're talking about introducing multi-minute 
> delays into the startup times of hundreds of millions of systems, regardless 
> of whether they have a proximate requirement for cryptographically-secure RNG 
> sources.

No I'm not. I'm talking about implementing this fix so that it *only* affects 
the Python interpreter start up, in particular things like SipHash 
initialization, instead of affecting the security of every other user of 
os.urandom.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Alex Gaynor]

Alex Gaynor added the comment:

Colm -- how is that situation not addressed by fixing the hash seed generation 
specifically, rather than patching all consumers of os.urandom?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Colm Buckley]

Colm Buckley added the comment:

Donald -

With the greatest respect, you're talking about introducing multi-minute delays 
into the startup times of hundreds of millions of systems, regardless of 
whether they have a proximate requirement for cryptographically-secure RNG 
sources. I don't think that's reasonable. My servers start up in about fifteen 
seconds with this patch applied, or over two minutes without.

Note; it's perfectly possible for getrandom() to block *indefinitely* - in the 
trigger case here (systemd's crontab generator), it times out after 90 seconds 
rather than eventually succeeding. If (for example), a Python script is called 
before device initialization, it's quite possible that there will *never* be 
enough entropy in the system to satisfy getrandom(), resulting in a non-booting 
system.

To reiterate; the overwhelming majority of applications (in particular, 
anything which is called after the entropy pool is initialized, which typically 
happens once networking, USB etc. are running) will use perfectly acceptable 
random sources. The only applications affected by this patch are those which 
call getrandom() very early in the boot process.

I feel you're tilting at a very impractical windmill.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com