[Security-sig] PEP 522: Allow BlockingIOError in security sensitive APIs on Linux

consider retaining consistency with "man urandom" to be a useful design goal (any more than the builtin open tries to retain consistency with "man open") Cheers, Nick. = PEP: 522 Title: Allow BlockingIOError in security sensitive APIs on Linux Version: $Revision$

Re: [Security-sig] RFC: PEP: Make os.urandom() blocking on Linux

may never realise. If your system hangs, you might eventually figure it out, but only after a likely frustrating debugging effort. By contrast, if your application fails with "BlockingIOError: system random number generator not ready", then you can search for that on the internet, see

Re: [Security-sig] RFC: PEP: Make os.urandom() blocking on Linux

On 23 June 2016 at 06:54, Donald Stufft wrote: > >> On Jun 23, 2016, at 8:48 AM, Barry Warsaw wrote: >> >> On Jun 22, 2016, at 06:31 PM, Nick Coghlan wrote: >> >>> try: >>> my_random = os.getrandom >>> except AttributeError: >

Re: [Security-sig] Can /dev/urandom ever revert from the "good" to the "bad" state?

traightforward "if you get this exception when running on Python 3.6, assess your application's needs, then apply one of these remedies" note for the Python 3.6 porting guide. Regards, Nick. [1] https://mail.python.org/pipermail/python-dev/2016-June/145146.html -- Nick Coghlan |

Re: [Security-sig] RFC: PEP: Make os.urandom() blocking on Linux

On 23 June 2016 at 10:31, Nick Coghlan wrote: > Reasonable developer experience: > > * just keep using os.urandom(), Python will transparently upgrade your > code to the best non-blocking-in-practice system interface the OS has > to offer > * if os.urandom() throws BlockingIOE

Re: [Security-sig] RFC: PEP: Make os.urandom() blocking on Linux

On 23 June 2016 at 11:13, Donald Stufft wrote: > >> On Jun 23, 2016, at 2:10 PM, Nick Coghlan wrote: >> >> That second one has the added bonus of doing the right thing even on >> older Linux kernels that don't provide the new getrandom() syscall, >> creatin

Re: [Security-sig] PEP 522: Allow BlockingIOError in security sensitive APIs on Linux

assume that ``/dev/urandom`` on >> Linux will always retain its current behaviour can be updated to access >> ``/dev/urandom`` directly:: >> >> def dev_urandom(num_bytes): >> with open("/dev/urandom", "rb") as f: >> return f.read(num_bytes) > > Again, I'm against adding such complexity for a corner case. Just use > os.urandom(). All of this would be triggered by *application* developers actually hitting the BlockingIOError and decide it was the appropriate course of application for *their* application. The point of this part of the PEP is to highlight that there are some really simple 3-5 functions that let developers get a wide variety of behaviours in ways that are compatible with single-source Python 2/3 code. >> For additional background details beyond those captured in this PEP, also see >> Victor Stinner's summary at http://haypo-notes.readthedocs.io/pep_random.html > > Oh, I didn't expect to have references to my document :-) I moved it to: > https://haypo-notes.readthedocs.io/summary_python_random_issue.html > > http://haypo-notes.readthedocs.io/pep_random.html is now really a PEP ;-) Cool, I'll update the first reference and also and a reference to your draft PEP. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Security-SIG mailing list Security-SIG@python.org https://mail.python.org/mailman/listinfo/security-sig

Re: [Security-sig] RFC: PEP: Make os.urandom() blocking on Linux

On 23 June 2016 at 11:38, Nick Coghlan wrote: > Although now I'm wondering whether it might be worth proposing a > "secrets.wait_for_system_rng()" API as part of PEP 522, with the > following implementation: > > def wait_for_system_rng(): > # Avoi

Re: [Security-sig] PEP 522: Allow BlockingIOError in security sensitive APIs on Linux

On 23 June 2016 at 17:46, Donald Stufft wrote: > >> On Jun 23, 2016, at 8:33 PM, Nick Coghlan wrote: >> >> The argument chain runs: >> >> - if such software doesn't exist, it doesn't matter which behaviour we choose >> - if we'

Re: [Security-sig] PEP: Make os.urandom() blocking on Linux (version 2)

m() may hang the world on Linux system boot' in the Python 3.6 porting notes". I'll still keep iterating on PEP 522, but I'm to the point of being +0 on this approach if Guido decides he prefers it :) Cheers, Nick. P.S. DevNation/Red Hat Summit are on next week, so I

Re: [Security-sig] PEP 522: Allow BlockingIOError in security sensitive APIs on Linux

once first, before falling back to blocking on /dev/random. Once the system RNG is ready, that first call will always succeed, no matter how many new processes you start. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___

Re: [Security-sig] PEP 522: Allow BlockingIOError in security sensitive APIs on Linux

ence may block waiting for the system RNG if you run it before the system RNG is ready The default state of Python 3.6+ applications would be the first one, and I think that's an entirely reasonable default - if you're writing userspace code that runs before the system RNG is ready, y

Re: [Security-sig] RFC: PEP: Make os.urandom() blocking on Linux

uncertainty and sensitive secrets?" that motivated the introduction of the secrets module isn't really applicable. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Security-SIG mailing list Security-SIG@python.org https://mail.python.org/mailman/listinfo/security-sig

Re: [Security-sig] PEP 522: Allow BlockingIOError in security sensitive APIs on Linux

On 24 June 2016 at 16:21, Victor Stinner wrote: > 2016-06-24 22:05 GMT+02:00 Nick Coghlan : >> As such, the idioms I currently have in PEP 522 are wrong - the "wait >> for the system RNG or not" decision wouldn't be one to be made on a >> per-call basis, bu

Re: [Security-sig] Policy PEP (was Re: RFC: PEP: Make os.urandom() blocking on Linux)

nuinely care about secure secret generation should upgrade to Python 3.6 and a Linux kernel with getrandom() support" is an entirely reasonable position for us to take in this case. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___

[Security-sig] getrandom() syscalls and Python runtime binary portability

d of /dev/random. Cheers, Nick. [1] https://hg.python.org/cpython/file/default/Python/random.c#l119 -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Security-SIG mailing list Security-SIG@python.org https://mail.python.org/mailman/listinfo/security-sig

[Security-sig] Does the buildtime HAVE_GETRANDOM_SYSCALL check actually make sense?

RANDOM_SYSCALL that looks for some other indicator that this is a build for a platform where getrandom() might be available at runtime, even if it's not available at build time. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia __

[Security-sig] PEP 522 (v2): Allow BlockingIOError in security sensitive APIs

ld systems work. Cheers, Nick. == PEP: 522 Title: Allow BlockingIOError in security sensitive APIs Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan , Nathaniel J. Smith Status: Draft Type: Standards Track Content-Type: text/x-rst Requires: 506 Created: 16

Re: [Security-sig] Does the buildtime HAVE_GETRANDOM_SYSCALL check actually make sense?

er glibc, so not worrying about this until it's resolved there sounds fine to me. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Security-SIG mailing list Security-SIG@python.org https://mail.python.org/mailman/listinfo/security-sig

Re: [Security-sig] Does the buildtime HAVE_GETRANDOM_SYSCALL check actually make sense?

On 2 July 2016 at 23:30, Nick Coghlan wrote: > On 26 June 2016 at 02:30, Victor Stinner wrote: >> The configure check ensures that constants required to build random.c are >> available. We can only run this check at the compilation. I don't want to >> maintain hard

Re: [Security-sig] PEP 524: Make os.urandom() blocking on Linux (version 3)

put the cross-platform blocking behaviour: either doing it implicitly in os.urandom, or letting that sometimes throw BlockingIOError, and offering an explicit cross-platform blocking API in the secrets module. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia __

Re: [Security-sig] PEP 524: Make os.urandom() blocking on Linux (version 3)

e them fail noisily, with a cross-platform way to opt-in to blocking (PEP 522) * make them "just work" in most cases, but potentially block in some situations (PEP 524) So what I'll probably do is update PEP 522 to assume os.getrandom() will be available (with the above rationale), and

Re: [Security-sig] PEP 524: Make os.urandom() blocking on Linux (version 3)

all fails. I'd say in the non-EINTR case with GRNG_RANDOM it would be reasonable to return a short read. So folks using that flag would need a Python-level loop, but folks using the kernel's CSPRNG wouldn't. Regards, Nick. -- Ni

Re: [Security-sig] PEP 524: Make os.urandom() blocking on Linux (version 3)

On 30 July 2016 at 02:52, Victor Stinner wrote: > 2016-07-29 17:37 GMT+02:00 Nick Coghlan : >>> Ok, but imagine that you use the "expensive" GRNG_RANDOM (/dev/random >>> rather than /dev/urandom). A first call returns 4000 bytes of "high >>> quality

Re: [Security-sig] Take a decision for os.urandom() in Python 3.6

of entropy) can just catch BlockingIOError, rather than needing to use platform specific APIs like os.getrandom(). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Security-SIG mailing list Security-SIG@python.org https://mail.python.org/mailman/listinfo/security-sig

Re: [Security-sig] Take a decision for os.urandom() in Python 3.6

On 6 August 2016 at 18:46, Nick Coghlan wrote: > On 6 August 2016 at 18:32, Victor Stinner wrote: >> Le 6 août 2016 04:39, "Guido van Rossum" a écrit : >>> 4. Bad code gets cargo-culted (e.g. through StackOverflow). >> >>> Re (4): With PEP 524, peo

Re: [Security-sig] Take a decision for os.urandom() in Python 3.6

notification), the latter design is the one I ended up converging on: high level API with implicit blocking, low level API that never blocks, but may throw an exception. It's not where I expected to end up when I first wrote the PEP, but that's the PEP process for you :) Cheers, Nick. >

Re: [Security-sig] Take a decision for os.urandom() in Python 3.6

there's nothing wrong with your application, but there *is* something wrong with your environment (since security sensitive Python code should only be run after the system RNG is ready) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Security-SIG mailing list Security-SIG@python.org https://mail.python.org/mailman/listinfo/security-sig

Re: [Security-sig] Take a decision for os.urandom() in Python 3.6

On 8 August 2016 at 02:14, Nick Coghlan wrote: > In a lot of ways, I see it as being similar to our dependency on the Linux > platform locale being set correctly to get boundary processing right: if > you get an exception, the problem *isn't* generally with the application, >

Re: [Security-sig] Take a decision for os.urandom() in Python 3.6

f gathering objective data about the scope of the problem from ABRT (Fedora's automatic bug reporting tool, which can automatically collect and submit Python stack traces, but can't readily detect system hangs). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Austr

Re: [Security-sig] Take a decision for os.urandom() in Python 3.6

On 8 August 2016 at 12:37, Nick Coghlan wrote: > However, from an operations perspective, it means the discussion will move > downstream to see whether we (Fedora) agree this is the right behaviour for > the *system* Python, or whether we should patch that to throw the error >

Re: [Security-sig] Take a decision for os.urandom() in Python 3.6

On 8 August 2016 at 13:32, Nick Coghlan wrote: > The Fedora Rawhide experiment I'm proposing in that email to the Fedora > Python list should give us the data we (Fedora) need to decide whether or > not this is one of those cases where it makes sense for us to carry a patch >

Re: [Security-sig] Take a decision for os.urandom() in Python 3.6

On 8 August 2016 at 14:22, Nick Coghlan wrote: > On 8 August 2016 at 13:32, Nick Coghlan wrote: > >> The Fedora Rawhide experiment I'm proposing in that email to the Fedora >> Python list should give us the data we (Fedora) need to decide whether or >> not this i

Re: [Security-sig] Take a decision for os.urandom() in Python 3.6

elling reason to address that build environment discrepancy for 3.5, but we (Fedora) are going to have to do something about it for Python 3.6 in F26 so that os.getrandom() gets defined properly and os.urandom() can be made blocking (with a warning when it does). Cheers, Nick. -- Nick Coghlan |

Re: [Security-sig] Pending security features for 3.6

ython 3.5 and 2.7, we're probably due for a successor to PEP 466 that syncs the ssl support in those versions with the 3.6 version of the module - that should hopefully be less controversial this time around. Cheers, Nick. -- Nick Coghlan

Re: [Security-sig] Implementation of the PEP 524

case, but I'd expect the "blocking" part to be the main delay there, rather than the second syscall. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Security-SIG mailing list Security-SIG@python.org https://mail.python.org/mailman/listinfo/security-sig

Re: [Security-sig] Implementation of the PEP 524

On 18 August 2016 at 03:16, Victor Stinner wrote: > 2016-08-17 18:39 GMT+02:00 Nick Coghlan : >> - if blocking *is* set, it prints a warning to stderr, before trying >> again without the GRND_NONBLOCK flag > > I implemented the PEP 524. Such warning is not part of the PEP. &

Re: [Security-sig] Implementation of the PEP 524

On 18 August 2016 at 22:10, Nick Coghlan wrote: > On 18 August 2016 at 03:16, Victor Stinner wrote: >> 2016-08-17 18:39 GMT+02:00 Nick Coghlan : >>> - if blocking *is* set, it prints a warning to stderr, before trying >>> again without the GRND_NONBLOCK flag >> &

Re: [Security-sig] Implementation of the PEP 524

On 18 August 2016 at 22:10, Nick Coghlan wrote: > OK, I'll review it on that basis, and then submit a follow-up RFE and > patch to add the warning. > > I'd like to minimise the divergence between upstream and Fedora, and > we'll at least have the warning in the dow

Re: [Security-sig] Unified TLS API for Python

;TLSv1_3" MAXIMUM_SUPPORTED = TLSv1_3 ``` That way folks get sensible answers regardless of whether they reference the enum entry name or its value, or render it directly with repr() or str(): >>> TLSVersion.MINIMUM_SUPPORTED >>> TLSVersion.MAXIMUM_SUPPOR

Re: [Security-sig] Unified TLS API for Python

oint the question becomes whether or not they're offering any benefit beyond just using "None" in the appropriate location. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Security-SIG mailing list

Re: [Security-sig] Unified TLS API for Python

SecureTransport, SChannel), and hopefully a few other implementations as well (e.g. NSS, BoringSSL). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Security-SIG mailing list Security-SIG@python.org https://mail.py

Re: [Security-sig] Unified TLS API for Python

sufficient to enforce minimum TLS versions and the acceptable cipher suites, while still leaving room for more capable third party policy enforcement. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Security-SIG mailing list Security-SIG@python.org https://mail.python.org/mailman/listinfo/security-sig

Re: [Security-sig] Unified TLS API for Python

On 13 January 2017 at 18:27, Cory Benfield wrote: > >> On 13 Jan 2017, at 08:04, Nick Coghlan wrote: >> >> I'd still be inclined to keep that concern separate from those of the >> core TLS context (which is about applying a given security context to >> so

Re: [Security-sig] Unified TLS API for Python

re often easier to test, since you can more readily isolate the effects of the individual operations 4. A public programmatic API serves as a natural constraint and test case for any subsequent declarative API Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, A

Re: [Security-sig] Unified TLS API for Python

entation TLSSocket: ABC to combine a context with a network socket TLSBuffer: ABC to combine a context with a pair of data buffers And then TLSPolicy would be a potential future implementation independent addition that could be used to constrain acceptable TLS configurations. Cheers, Nick.

Re: [Security-sig] Unified TLS API for Python

On 14 January 2017 at 01:58, Cory Benfield wrote: > >> On 13 Jan 2017, at 15:45, Nick Coghlan wrote: >> >> So the essential stack would look like: >> >> TLSConfig[uration?]: implementation independent, settings only >> TLSClientContext: ABC

Re: [Security-sig] Unified TLS API for Python

asonable learning curve as they move away from "the defaults are good enough for me" to "I need to make these changes, for these reasons", even as we attempt to make sure that "routine" use cases are secure by default. C

Re: [Security-sig] Unified TLS API for Python: Round 2

bclass these exceptions rather > than throw them directly. This makes it moderately easier to determine which > concrete TLS implementation is in use during debugging of unexpected errors. > However, this is not mandatory. This is the one part of the PEP that I think may need to discu

Re: [Security-sig] Unified TLS API for Python: Round 2

rder to support running in environments that don't provide an OpenSSL backend at all Option 5 would cover even that last case: legacy API consumers that only relied on being able to catch the legacy exceptions would tolerate the use of non-OpenSSL backends even in environments where OpenSSL i

Re: [Security-sig] Unified TLS API for Python: Round 2

On 26 January 2017 at 10:50, Cory Benfield wrote: > >> On 26 Jan 2017, at 07:49, Nick Coghlan wrote: >> >> Option 4: tls.TLSError, tls.WantReadError, tls.WantWriteError are >> defined as inheriting from ssl.SSLError, ssl.SSLWantReadError, and >> ssl.SSLWantWrite

Re: [Security-sig] email & phones

t when quoting bits selectively is a PITA. Basically, it's set up on the assumption that all recipients will be using a client that autohides quoted text, and only expands it when requested to do so. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane

Re: [Security-sig] Unified TLS API for Python: Round 2

On 26 January 2017 at 16:22, Cory Benfield wrote: > > On 26 Jan 2017, at 14:23, Nick Coghlan wrote: > > In an ideal world, that could be handled just by having the ssl module > import the new tls module and alias the exceptions accordingly. > Talking to Christian about it,

Re: [Security-sig] Patching ssl.py to workaround ssl lack of relocability

her potential locations reliable give a detectable error :( Given your context of use though, the one potential incompatibility you're going to have to watch out for is losing access to any custom CA certificates that are installed into the system trust stores (since certifi won't have any know

Re: [Security-sig] PEP 551: Security transparency in the Python runtime

ther than log hooks. I also think this could be a separate module at the Python level (e.g. "runtimeaudit"), and a separate prefix at the C API level (e.g. "PyAudit_*") rather than needing to be directly in the sys namespace. Cheers, Nick. -- Nick Coghlan | ncogh...@gmai

Re: [Security-sig] List Settings Question

e if we could at least stop digging the hole deeper by encouraging new lists to start out on MM3, and offering a way for list owners to request piecemeal migrations. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia __

[Security-sig] Re: Backport critical bugfixes?

Lib/test/test_crashers.py) and resolve the already known segfaults first. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Security-SIG mailing list security-sig@python.org https://mail.python.org/mm3/mailman3/lists/security-sig.python.org/