[issue27292] Warn users that os.urandom() can return insecure values

[Martin Panter]

Martin Panter added the comment:

Restored “On Linux” for the changed in 3.5.2 notice. I do think it is better to 
be general and future-proof, but that is a separate, less important issue to 
the main purpose of the patch. (I don’t know if Solaris’s version can block or 
not.)

--
stage: needs patch -> patch review
Added file: http://bugs.python.org/file43442/urandom-doc.v2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27292] Warn users that os.urandom() can return insecure values

[Donald Stufft]

Changes by Donald Stufft :


--
nosy:  -dstufft

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27292] Warn users that os.urandom() can return insecure values

[STINNER Victor]

STINNER Victor added the comment:

> As far as I can see (looking at Python/random.c and configure.ac), the 
> Solaris version should also use GRND_NONBLOCK:

Oh, you're right: I didn't notice that GRND_NONBLOCK was also used on Solaris. 
The change is not deliberate, but it is good to do that :-)

Solaris getrandom() is documented to fail with EAGAIN if "No entropy is 
available and GRND_NONBLOCK is set."
https://docs.oracle.com/cd/E53394_01/html/E54765/getrandom-2.html

The question is more if reading from /dev/urandom block in this case.

If we don't know, I would prefer to keep the "On Linux" prefix in the doc, and 
don't say anything about Solaris.

I'm able to check the behaviour of Solaris.

You should contact the developers who get access to Solaris, you can meet them 
in the previous random issues specific to Solaris: issue #25003 and issue 
#26735.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27292] Warn users that os.urandom() can return insecure values

[Martin Panter]

Martin Panter added the comment:

Rebased so Rietveld can work with it, earlier version was my fault.

As far as I can see (looking at Python/random.c and configure.ac), the Solaris 
version should also use GRND_NONBLOCK:

#ifdef MS_WINDOWS
#elif defined(HAVE_GETENTROPY) && !defined(sun)
#else

#if defined(HAVE_GETRANDOM) || defined(HAVE_GETRANDOM_SYSCALL)
const int flags = GRND_NONBLOCK;
#ifdef HAVE_GETRANDOM
n = getrandom(dest, n, flags);
#else
n = syscall(SYS_getrandom, dest, n, flags);
#endif

Apart from using a C function call versus syscall(), I don’t see there is much 
difference between the Solaris and Linux cases. Correct me if I’m wrong though.

--
Added file: http://bugs.python.org/file43411/urandom-doc.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27292] Warn users that os.urandom() can return insecure values

[STINNER Victor]

STINNER Victor added the comment:

Strange, I don't see the [Review] button.

.. versionchanged:: 3.5.2
-  On Linux, if ``getrandom()`` blocks (the urandom entropy pool is not
+  If ``getrandom()`` blocks (the urandom entropy pool is not
   initialized yet), fall back on reading ``/dev/urandom``.

Please keep "On Linux", getrandom() is also used on Solaris, and my change is 
really restricted to Linux.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27292] Warn users that os.urandom() can return insecure values

[Martin Panter]

Martin Panter added the comment:

Here is a possible patch for 3.5+ based on my modest understanding of the 
concerns about insecure results and blocking. I hope that my wording is clear, 
couldn’t be confused with Linux’s /dev/random blocking and running out of fresh 
entropy, etc.

I also tried to make it clearer what APIs are used in what circumstances. It is 
not just Linux: we also call getrandom() on Solaris, because its getentropy() 
is not good enough.

--
keywords: +patch
Added file: http://bugs.python.org/file43410/urandom-doc.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27292] Warn users that os.urandom() can return insecure values

[Martin Panter]

Martin Panter added the comment:

As far as this bug goes, 3.5 is not very different from 2.7

--
nosy: +martin.panter
versions: +Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27292] Warn users that os.urandom() can return insecure values

[Christian Heimes]

Changes by Christian Heimes :


--
nosy:  -christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27292] Warn users that os.urandom() can return insecure values

[Larry Hastings]

Larry Hastings added the comment:

This is not a release blocker.

--
priority: release blocker -> normal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27292] Warn users that os.urandom() can return insecure values

[Larry Hastings]

Larry Hastings added the comment:

I would suggest weakening the one-line summary.  Currently the first line reads:

Return a string of n random bytes suitable for cryptographic use.

I'd support adding some "weasel words" to this, e.g.:

Return a string of n random bytes that should be suitable
for cryptographic use.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27292] Warn users that os.urandom() can return insecure values

[Donald Stufft]

Donald Stufft added the comment:

I agree that we should add a warning to these, it's easy to see how someone 
might read the summary of the function "Return a string of n random bytes 
suitable for cryptographic use." and skip over the nuance in the rest of the 
body of the function. Adding a red box to ensure that they know that on popular 
platforms os.urandom is not going to always return bytes that are suitable for 
cryptographic use is pretty important in my opinion. The current wording makes 
it sound like it's something you only need to worry about on "weird" platforms, 
not on one of the (if not the) most popular platforms for running Python on.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27292] Warn users that os.urandom() can return insecure values

[Larry Hastings]

Larry Hastings added the comment:

Oh, and, for 3.6 I would definitely support adding a mention of "Instead of 
using this function directly, we recommend you use the token_bytes() function 
in the secrets module", blah blah best practices etc.

That goes for os.getrandom() too, if we add it to 3.6 (which I rather suspect 
we will).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27292] Warn users that os.urandom() can return insecure values

[Larry Hastings]

Larry Hastings added the comment:

I don't think this is necessary, as the documentation for os.urandom() is 
already pretty good.  Here's the relevant bit:

This function returns random bytes from an OS-specific randomness
source. The returned data should be unpredictable enough for
cryptographic applications, though its exact quality depends on
the OS implementation. On a Unix-like system this will query
/dev/urandom, and on Windows it will use CryptGenRandom().

ISTM that the Python documentation doesn't generally indulge in warning users 
about specific shortcomings of particular platforms; if it did it would be 
littered with such warnings.

Personally I'd approve of making the existing statements a little more 
forceful, like pulling it out into a red "warning" box and making it explicit 
that os.urandom() isn't any more sophisticated than the platform-specific 
technologies it uses.  But that's as far as I'd go.  I wouldn't add all the 
specifics you suggest.

Technically I think this actually is my call, as I'm the "platform expert" for 
the posix module:
  https://docs.python.org/devguide/experts.html
But really I think it's the call of the "Documentation Expert" for the relevant 
releases.  This is a stylistic concern--should the Python docs delve into these 
sorts of details?--and that's really the domain of the DE.

Georg Brandl is the DE for all currently-supported versions of Python.  (Well, 
2.7 has no official DE, but I think Georg is de facto DE for that release too.) 
 I've nosied him here; hopefully he can tell us the standard Python doc 
aesthetic when it comes to these sorts of concerns.


By the way, the Raspberry PI does have hardware RNG:
  
http://scruss.com/blog/2013/06/07/well-that-was-unexpected-the-raspberry-pis-hardware-random-number-generator/
It required loading an extra driver, at least as of 2014.  I concede I don't 
know what current crypto best-practices are on the PI.

That's one good reason why I think the Python documentation doesn't indulge in 
these laundry lists of platform failings--such information has a tendency to 
become out-of-date without anyone noticing.

--
nosy: +georg.brandl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27292] Warn users that os.urandom() can return insecure values

[Cory Benfield]

Changes by Cory Benfield :


--
nosy: +Lukasa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27292] Warn users that os.urandom() can return insecure values

[Steven D'Aprano]

Steven D'Aprano added the comment:

Relevant: issue #27293

(I've taken the liberty of subscribing those on this issues nosy list to the 
new issue, I hope that's okay)

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27292] Warn users that os.urandom() can return insecure values

[Christian Heimes]

New submission from Christian Heimes:

In #26839 os.urandom() was made non-blocking and non-exception-raising on 
Linux. As a result os.urandom() is no longer a CSPRNG under some conditions as 
it can and will return predictable random values without any sort of warning or 
error flag. These conditions are (including but not limited to):

* early boot state
* virtualization without host-passthrough, e.g. virtio-rng
* embedded devices without hardware RNG or RTC, e.g. raspberry pi

Please ensure that the documentation properly warns users about these edge 
cases.

We might also want to add that os.urandom() can block on other platforms, e.g. 
FreeBSD and OpenBSD.

--
components: Library (Lib)
keywords: 3.5regression
messages: 268205
nosy: christian.heimes, dstufft, haypo, larry, ned.deily
priority: release blocker
severity: normal
stage: needs patch
status: open
title: Warn users that os.urandom() can return insecure values
type: security
versions: Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com