Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

I understood that Christian Heimes and/or Donald Stufft are interested to work on a PEP. 2016-06-09 13:25 GMT+02:00 Larry Hastings : > A problem has surfaced just this week in 3.5.1. Obviously this is a good > time to fix it for 3.5.2. But there's a big argument over what is

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

Excerpts from R. David Murray's message of 2016-06-09 08:41:01 -0400: > On Thu, 09 Jun 2016 13:12:22 +0100, Cory Benfield wrote: > > The Linux kernel can���t change this stuff easily because they mustn���t > > break userspace. Python *is* userspace, we can do what we like, and

[Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

A problem has surfaced just this week in 3.5.1. Obviously this is a good time to fix it for 3.5.2. But there's a big argument over what is "broken" and what is an appropriate "fix". As 3.5 Release Manager, I can put my foot down and make rulings, and AFAIK the only way to overrule me is

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On Thu, 09 Jun 2016 13:12:22 +0100, Cory Benfield wrote: > The Linux kernel can’t change this stuff easily because they mustn’t > break userspace. Python *is* userspace, we can do what we like, and we I don't have specific input on the rest of this discussion, but I

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

> On Jun 9, 2016, at 7:25 AM, Larry Hastings wrote: > > A problem has surfaced just this week in 3.5.1. Obviously this is a good > time to fix it for 3.5.2. But there's a big argument over what is "broken" > and what is an appropriate "fix". Couple clarifications:

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

> On Jun 9, 2016, at 7:25 AM, Larry Hastings wrote: > > 6) Guido and Tim Peters already decided once that os.urandom() should behave > like /dev/urandom. > > Issue #25003: > http://bugs.python.org/issue25003 To be exceedingly clear, in

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On 2016-06-09 13:25, Larry Hastings wrote: > > A problem has surfaced just this week in 3.5.1. Obviously this is a > good time to fix it for 3.5.2. But there's a big argument over what is > "broken" and what is an appropriate "fix". > > As 3.5 Release Manager, I can put my foot down and make

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

> On 9 Jun 2016, at 12:54, Christian Heimes wrote: > > Therefore I propose to fix problem 2 and 3: > > - add a new random_seed member to _Py_HashSecret and use it to derive an > initial Mersenne-Twister state for the default random instance of the > random module. > > -

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

> On Jun 9, 2016, at 9:27 AM, Cory Benfield wrote: > > >> On 9 Jun 2016, at 13:53, Doug Hellmann wrote: >> >> I agree with David. We shouldn't break existing behavior in a way >> that might lead to someone else's software being unusable. > > What

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

> On Jun 9, 2016, at 9:48 AM, Doug Hellmann wrote: > > All of which fails to be backwards compatible (new exceptions and hanging > behavior), which means you’re breaking apps. Introducing a new API lets the > developers who care about strong random values use them

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

Larry Hastings wrote: On 3.4 and before, on Linux, os.urandom() would never block, but if the > entropy pool was uninitialized it could return very-very-poor-quality > random bits. On 3.5.0 and 3.5.1, on Linux, when using the getrandom() > call, it will instead block for an apparently unbounded

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

> On Jun 9, 2016, at 11:52 AM, Guido van Rossum wrote: > > Wow. I have to decide an issue on which lots of people I respect disagree > strongly. So no matter how I decide some of you are going to hate me. Oh > well. :-( > > So let's summarize the easy part first. It seems

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On 06/09/2016 04:25 AM, Larry Hastings wrote: A problem has surfaced just this week in 3.5.1. Obviously this is a good time to fix it for 3.5.2. But there's a big argument over what is "broken" and what is an appropriate "fix". Having read the thread thus far, here is my take on fixing it:

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

Wow. I have to decide an issue on which lots of people I respect disagree strongly. So no matter how I decide some of you are going to hate me. Oh well. :-( So let's summarize the easy part first. It seems that there is actually agreement that for the initialization of hash randomization and for

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On Thu, Jun 09, 2016 at 12:39:00PM -0400, Donald Stufft wrote: > There are three options for what do with os.urandom by default: > > * Allow it to silently return data that may or may not be > cryptographically secure based on what the state of the urandom pool > initialization looks like.

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On 9 June 2016 at 17:54, Ben Leslie wrote: >> My opinion is that blocking is slightly better than raising an exception >> because it matches what other OSs do, but that both blocking and raising an >> exception is better than silently giving data that may or may not be >>

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

> On Jun 9, 2016, at 1:21 PM, Paul Moore wrote: > > On 9 June 2016 at 17:54, Ben Leslie wrote: >>> My opinion is that blocking is slightly better than raising an exception >>> because it matches what other OSs do, but that both blocking and raising an

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

> On Jun 9, 2016, at 1:14 PM, Steven D'Aprano wrote: > > On Thu, Jun 09, 2016 at 12:39:00PM -0400, Donald Stufft wrote: > >> There are three options for what do with os.urandom by default: >> >> * Allow it to silently return data that may or may not be >>

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On Thu, Jun 09, 2016 at 12:54:31PM -0400, Ben Leslie wrote: > I think an exception is much easier for a user to deal with from a > practical point of view. Trying to work out why a process has hung is > obviously possible, but not necessarily easy. > > Having a process crash due to an exception

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On Thu, Jun 09, 2016 at 08:26:20AM -0400, Donald Stufft wrote: > random.py > - > > In the abstract it doesn't hurt to seed MT with a CSPRNG, it just doesn't > provide much (if any) benefit and in this case it is hurting us because of the > cost on import (which will exist on other

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On 9 June 2016 at 12:39, Donald Stufft wrote: > >> On Jun 9, 2016, at 12:30 PM, Steven D'Aprano wrote: >> >>> >>> os.urandom >>> -- >> [...] >>> With that in mind, I think that we should, to the best of our ability given >>> the >>> platform we're

Re: [Python-Dev] PEP 468

> From: zr...@fastmail.com > Subject: [Python-Dev] PEP 468 > > Is there any further thoughts on including this in 3.6? Similar to the > recent discussion on OrderedDict namespaces for metaclasses, this would > simplify / enable a number of type factory use cases where proper > metaclasses are

[Python-Dev] PEP 468

Is there any further thoughts on including this in 3.6? Similar to the recent discussion on OrderedDict namespaces for metaclasses, this would simplify / enable a number of type factory use cases where proper metaclasses are overkill. This feature would also be quite nice in say pandas where the

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On 9 June 2016 at 04:25, Larry Hastings wrote: > A user reports that when starting CPython soon after startup on a fresh > virtual machine, the process would hang for a long time. Someone on the > issue reported observed delays of over 90 seconds. Later we found out: it >

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On Thu, Jun 09, 2016 at 06:21:32PM +0100, Paul Moore wrote: > If we put the specific issue of applications that run very early in > system startup to one side, is there a possibility of running out of > entropy during normal system use? Even for a tiny duration? With /dev/urandom, I believe the

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On 9 June 2016 at 13:29, Steven D'Aprano wrote: > On Thu, Jun 09, 2016 at 12:54:31PM -0400, Ben Leslie wrote: > >> I think an exception is much easier for a user to deal with from a >> practical point of view. Trying to work out why a process has hung is >> obviously

Re: [Python-Dev] Smoothing the transition from Python 2 to 3

On 06/08/2016 02:40 PM, Fred Drake wrote: On Wed, Jun 8, 2016 at 5:33 PM, Ryan Gonzalez wrote: What about something like "unpythonic" or similar? Or perhaps... antipythy? That's awfully close to antipathy [1], my path module on PyPI. Besides, I liked the suggestion from

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On 06/09/2016 03:44 PM, Ethan Furman wrote: On 06/09/2016 03:22 PM, Larry Hastings wrote: Okay, it's decided: os.urandom() must be changed for 3.5.2 to never block on a getrandom() call. One way to not block is to raise an exception. Since this is such a rare occurrence anyway I don't see

Re: [Python-Dev] Smoothing the transition from Python 2 to 3

On 2016-06-09, Brett Cannon wrote: > I don't think you meant for what you said to sound insulting, > Neil, but it did feel like it upon first reading. Sorry, I think I misunderstood what you and Nick were saying. I've experienced a fair amount of negative feedback on my idea so I'm pretty cranky

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

I don't think we should add a new function. I think we should convince ourselves that there is not enough of a risk of an exploit even if os.urandom() falls back. On Thu, Jun 9, 2016 at 6:03 PM, Nathaniel Smith wrote: > On Thu, Jun 9, 2016 at 3:22 PM, Larry Hastings

Re: [Python-Dev] BDFL ruling request: should we block foreverwaiting for high-quality random bits?

(fat fingered the send button, picking up where I left off) If the pattern is really going to be the hasattr check you posted earlier, can we just do it for people and save them writing code that won't work on different OSs? Cheers, Steve Top-posted from my Windows Phone -Original

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On Thu, Jun 9, 2016 at 3:22 PM, Larry Hastings wrote: > > On 06/09/2016 08:52 AM, Guido van Rossum wrote: > >> That leaves direct calls to os.urandom(). I don't think this should block >> either. > > > Then it's you and me against the rest of the world ;-) > > > Okay, it's

Re: [Python-Dev] Smoothing the transition from Python 2 to 3

On 2016-06-09, Brett Cannon wrote: > On Thu, 9 Jun 2016 at 14:56 Nick Coghlan wrote: > > Once you switch to those now recommended more conservative migration > > tools, the tool suite you request already exists: > > > > - update your code with modernize or futurize > > - check

Re: [Python-Dev] Smoothing the transition from Python 2 to 3

On Thu, 9 Jun 2016 at 16:08 Neil Schemenauer wrote: > On 2016-06-09, Brett Cannon wrote: > > On Thu, 9 Jun 2016 at 14:56 Nick Coghlan wrote: > > > Once you switch to those now recommended more conservative migration > > > tools, the tool suite you request

Re: [Python-Dev] BDFL ruling request: should we block foreverwaiting for high-quality random bits?

Can we get any new function on all platforms, deferring to urandom() if getrandom() isn't there? If the pattern is really going to be the hasattr check you posted earlier Top-posted from my Windows Phone -Original Message- From: "Larry Hastings" Sent: ‎6/‎10/‎2016

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

Steven D'Aprano wrote: - Linux /dev/urandom doesn't block, but it might return predictable, poor-quality pseudo-random bytes (i.e. a potential exploit); - Other OSes may block for potentially many minutes (i.e. a potential DOS). It's even possible that it could block *forever*. There

Re: [Python-Dev] PEP 520: Ordered Class Definition Namespace

On 7 June 2016 at 20:17, Eric Snow wrote: > On Tue, Jun 7, 2016 at 6:20 PM, Ethan Furman wrote: >> If __definition_order__ is supposed to be immutable as well as read-only >> then we should convert non-tuples to tuples. No point in letting that

Re: [Python-Dev] Smoothing the transition from Python 2 to 3

On Thu, 9 Jun 2016 at 14:56 Nick Coghlan wrote: > On 8 June 2016 at 14:01, Neil Schemenauer wrote: > > [I've posted something about this on python-ideas but since I now > > have some basic working code, I think it is more than an idea.] > > > > I think the

Re: [Python-Dev] Smoothing the transition from Python 2 to 3

On 8 June 2016 at 14:01, Neil Schemenauer wrote: > [I've posted something about this on python-ideas but since I now > have some basic working code, I think it is more than an idea.] > > I think the uptake of Python 3 is starting to accelerate. That's > good. However, there are

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On 06/09/2016 08:52 AM, Guido van Rossum wrote: That leaves direct calls to os.urandom(). I don't think this should block either. Then it's you and me against the rest of the world ;-) Okay, it's decided: os.urandom() must be changed for 3.5.2 to never block on a getrandom() call. It's

Re: [Python-Dev] Smoothing the transition from Python 2 to 3

On Thu, Jun 9, 2016 at 6:16 PM, Ethan Furman wrote: > That's awfully close to antipathy [1], my path module on PyPI. Good point. Increasing confusion would not help. > Besides, I liked the suggestion from the -ideas list: Python 2therescue. ;) Nice; I like that too. :-)

Re: [Python-Dev] PEP 520: Ordered Class Definition Namespace

On 06/09/2016 02:39 PM, Nick Coghlan wrote: On 7 June 2016 at 20:17, Eric Snow wrote: On Tue, Jun 7, 2016 at 6:20 PM, Ethan Furman wrote: If __definition_order__ is supposed to be immutable as well as read-only then we should convert non-tuples to tuples. No point in letting that user bug

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On 06/09/2016 10:22 AM, Donald Stufft wrote: On Jun 9, 2016, at 1:14 PM, Steven D'Aprano wrote: Just to be clear, this is only an option on Linux, right? All the other major platforms block, whatever we decide to do on Linux. Including Windows? To my knowledge, all other

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On 06/09/2016 03:22 PM, Larry Hastings wrote: On 06/09/2016 08:52 AM, Guido van Rossum wrote: That leaves direct calls to os.urandom(). I don't think this should block either. Then it's you and me against the rest of the world ;-) Okay, it's decided: os.urandom() must be changed for 3.5.2

Re: [Python-Dev] Smoothing the transition from Python 2 to 3

On Jun 09, 2016, at 05:35 PM, Neil Schemenauer wrote: >Amber Brown claimed that she spent $60k of her time porting Twisted to Python >3. I think there is lots of room to make our porting tools better. Amber gave a presentation at the language summit and a Pycon talk. The latter video is up on

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On Jun 09 2016, Guido van Rossum wrote: > I don't think we should add a new function. I think we should convince > ourselves that there is not enough of a risk of an exploit even if > os.urandom() falls back. That will be hard, because you have to consider an active, clever

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On Thu, Jun 9, 2016 at 6:53 PM, Barry Warsaw wrote: > On Jun 09, 2016, at 03:22 PM, Larry Hastings wrote: > >>On 06/09/2016 08:52 AM, Guido van Rossum wrote: >>> That leaves direct calls to os.urandom(). I don't think this should > block >>> either. >> >>Then it's you and me

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On 06/09/2016 07:38 PM, Nikolaus Rath wrote: On Jun 09 2016, Larry Hastings wrote: Nope, I want the old behavior back. os.urandom() should read /dev/random if getrandom() would block. As the British say, "it should do what it says on the tin". Aeh, what the tin says is

Re: [Python-Dev] BDFL ruling request: should we block foreverwaiting for high-quality random bits?

On 06/09/2016 05:00 PM, Steve Dower wrote: If the pattern is really going to be the hasattr check you posted earlier, can we just do it for people and save them writing code that won't work on different OSs? No. That's what got us into this mess in the first place. 3.5.0 and 3.5.1 *already*

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On Thu, Jun 9, 2016 at 8:11 PM, Larry Hastings wrote: > > On 06/09/2016 07:58 PM, Nathaniel Smith wrote: > > I suspect the crypto folks would be okay with pushing this back to > 3.6, so long as the final resolution is that os.urandom remains the > standard interface for, as

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On Jun 09, 2016, at 03:22 PM, Larry Hastings wrote: >On 06/09/2016 08:52 AM, Guido van Rossum wrote: >> That leaves direct calls to os.urandom(). I don't think this should > block >> either. > >Then it's you and me against the rest of the world ;-) FWIW, I agree with you and Guido. I'm also

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On Jun 09 2016, Larry Hastings wrote: > On 06/09/2016 07:38 PM, Nikolaus Rath wrote: >> On Jun 09 2016, Larry Hastings wrote: >>> Nope, I want the old behavior back. os.urandom() should read >>> /dev/random if getrandom() would block. As the British say,

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

So secrets.py needs an upgrade; it currently uses random.SysRandom. On Thursday, June 9, 2016, Tim Peters wrote: > [Nikolaus Rath] > >> Aeh, what the tin says is "return random bytes". > > [Larry Hastings] > > What the tin says is "urandom", which has local man pages that

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On Jun 09 2016, Larry Hastings wrote: > On 06/09/2016 03:44 PM, Ethan Furman wrote: >> On 06/09/2016 03:22 PM, Larry Hastings wrote: >>> Okay, it's decided: os.urandom() must be changed for 3.5.2 to never >>> block on a getrandom() call. >> >> One way to not block is to raise

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

[Nikolaus Rath] >> Aeh, what the tin says is "return random bytes". [Larry Hastings] > What the tin says is "urandom", which has local man pages that dictate > exactly how it behaves. On Linux the "urandom" man page says: > > A read from the /dev/urandom device will not block waiting for

Re: [Python-Dev] PEP: Ordered Class Definition Namespace

On 06/07/2016 11:13 AM, Eric Snow wrote: On Tue, Jun 7, 2016 at 11:01 AM, Ethan Furman wrote: On 06/07/2016 10:51 AM, Eric Snow wrote: Specification = * types for which `__prepare__()`` returned something other than ``OrderedDict`` (or a subclass)

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

> On 06/09/2016 08:52 AM, Guido van Rossum wrote: > That leaves direct calls to os.urandom(). I don't think this should block > either. On 9 June 2016 at 22:22, Larry Hastings wrote: > Then it's you and me against the rest of the world ;-) > > > Okay, it's decided:

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

> On 9 Jun 2016, at 14:48, Doug Hellmann wrote: > > I agree those are the two options. I want the application developer to make > the choice, not us. Right, but right now those two options aren’t available: only one of them is. And one way or another we’re taking an

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

> On Jun 9, 2016, at 8:53 AM, Doug Hellmann wrote: > > Excerpts from R. David Murray's message of 2016-06-09 08:41:01 -0400: >> On Thu, 09 Jun 2016 13:12:22 +0100, Cory Benfield wrote: >>> The Linux kernel can���t change this stuff easily because they

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

> On 9 Jun 2016, at 13:53, Doug Hellmann wrote: > > I agree with David. We shouldn't break existing behavior in a way > that might lead to someone else's software being unusable. What does ‘usable’ mean? Does it mean “the code must execute from beginning to end”? Or

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On 2016-06-09 19:14, Steven D'Aprano wrote: > On Thu, Jun 09, 2016 at 12:39:00PM -0400, Donald Stufft wrote: > >> There are three options for what do with os.urandom by default: >> >> * Allow it to silently return data that may or may not be >> cryptographically secure based on what the state of

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

To expand on my idea of printing a warning, in 3.6 we could add a new Warning exception for this purpose, so you'd have command-line control over the behavior of os.urandom() by specifying -WXXX on your Python command line. For 3.5.2 that's too fancy though -- we can't add a new exception. --

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

On 6/9/2016 9:48 AM, Doug Hellmann wrote: On Jun 9, 2016, at 9:27 AM, Cory Benfield wrote: The problem here is that both definitions of ‘broken’ are unclear. If we leave os.urandom() as it is, there is a small-but-nonzero change that your program will hang, potentially

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

> On Jun 9, 2016, at 12:30 PM, Steven D'Aprano wrote: > >> >> os.urandom >> -- > [...] >> With that in mind, I think that we should, to the best of our ability given >> the >> platform we're on, ensure that os.urandom does not return bytes that the OS >> does not