Thanks Cory for the long explanation. Let me try to summarize (tell me if I'm wrong).
We have 3 options: * Do nothing: reject the PEP 546 and let each project handles security on its own (current status co) * Write *new* C code, maybe using certitude as a starting point, to offload certifcate validation on Windows and macOS * Backport existing code from master to 2.7: MemoryBIO and SSLObject Writing new code seems more risky and error-prone than backporting already "battle-tested" MemoryBIO from master. I also expect that writing code to validate certificate will be longer than the "100 lines of C code in (probably)" expected by Steve Dower. rust-certitude counts around 700 lines of Rust and 80 lines of Python code. But maybe I misunderstood the purpose of certitude: Steve Dower asked to only validate a certificate, not load or export CA. I counted 150 Python lines for SSLObject and 230 C lines for MemoryBIO. Since the long term plan is to not use stdlib ssl but a new implementation on Windows and macOS, it seems worthless to backport MemoryBIO on Python 2.7. The PEP 546 (backport MemoryBIO) is a practical solution to provide a *smooth* transition from ssl to a new TLS API. The experience showed that hard changes like "run 2to3 and drop your Python 2 code" doesn't work in practice. Users want a transition plan with small steps. Victor 2017-06-02 11:08 GMT+02:00 Cory Benfield <c...@lukasa.co.uk>: > > On 1 Jun 2017, at 20:59, Steve Dower <steve.do...@python.org> wrote: > > On 01Jun2017 1010, Nathaniel Smith wrote: > > I believe that for answering this question about the ssl module, it's really > only Linux users that matter, since pip/requests/everyone else pushing for > this only want to use ssl.MemoryBIO on Linux. Their plan on Windows/MacOS > (IIUC) is to stop using the ssl module entirely in favor of new ctypes > bindings for their respective native TLS libraries. > (And yes, in principle it might be possible to write new ctypes-based > bindings for openssl, but (a) this whole project is already teetering on the > verge of being impossible given the resources available, so adding any major > extra deliverable is likely to sink the whole thing, and (b) compared to the > proprietary libraries, openssl is *much* harder and riskier to wrap at the > ctypes level because it has different/incompatible ABIs depending on its > micro version and the vendor who distributed it. This is why manylinux > packages that need openssl have to ship their own, but pip can't and > shouldn't ship its own openssl for many hopefully obvious reasons.) > > > How much of a stop-gap would it be (for Windows at least) to override > OpenSSL's certificate validation with a call into the OS? This leaves most > of the work with OpenSSL, but lets the OS say yes/no to the certificates > based on its own configuration. > > For Windows, this is under 100 lines of C code in (probably) _ssl, and while > I think an SChannel based approach is the better way to go long-term,[1] > offering platform-specific certificate validation as the default in 2.7 is > far more palatable than backporting new public API. > > > It’s entirely do-able. This is where I reveal just how long I’ve been > fretting over this problem: https://pypi.python.org/pypi/certitude. Ignore > the description, it’s wildly out-of-date: let me summarise the library > instead. > > Certitude is a Python library that uses CFFI and Rust to call into the > system certificate validation libraries on macOS and Windows using a single > unified API. Under the covers it has a whole bunch of Rust code that > translates from what OpenSSL can give you (a list of certificates in the > peer cert chain in DER format) and into what those two operating systems > expect. The Rust code for Windows is here[1] and is about as horrifying a > chunk of Rust as you can imagine seeing (the Windows API does not translate > very neatly into Rust so the word “unsafe” appears a lot), but it does > appear to work, at least in the mainline cases and in the few tests I have. > The macOS code is here[2] and is moderately less horrifying, containing no > instances of the word “unsafe”. > > I lifted this approach from Chrome, because at the moment this is what they > do: they use their custom fork of OpenSSL (BoringSSL) to do the actual TLS > protocol manipulation, but hand the cert chain verification off to > platform-native libraries on Windows and macOS. > > I have never productised this library because ultimately I never had the > time to spend writing a sufficiently broad test-case to confirm to me that > it worked in all cases. There are very real risks in calling these APIs > directly because if you get it wrong it’s easy to fail open. > > It should be noted that right now certitude only works with, surprise, > PyOpenSSL. Partly this is because the standard library does not expose > SSL_get_peer_cert_chain, but even if it did that wouldn’t be enough as > OpenSSL with VERIFY_NONE does not actually *save* the peer cert chain > anywhere. That means even with PyOpenSSL the only way to get the peer cert > chain is to hook into the verify callback and save off the certs as they > come in, a gloriously absurd solution that is impossible with pure-Python > code from the ssl module. > > While this approach could work with _ssl.c, it ultimately doesn’t resolve > the issue. It involves writing a substantial amount of new code which needs > to be maintained by the ssl module maintainers. All of this code needs to be > tested *thoroughly*, because python-dev would be accepting responsibility > for the incredibly damaging potential CVEs in that code. And it doesn’t get > python-dev out of the business of shipping OpenSSL on macOS and Windows, > meaning that python-dev continues to bear the burden of OpenSSL CVEs, as > well as the brand new CVEs that it is at risk of introducing. > > Oh, and it can’t be backported to Python 2.7 or any of the bugfix-only > Python 3 releases, and as I just noted the ssl module has never made it > possible to use this approach from outside CPython. So it’s strictly just as > bad as the situation PEP 543 is in, but with more C code. Doesn’t sound like > a winning description to me. ;) > > Cory > > [1]: > https://github.com/Lukasa/rust-certitude/blob/master/rust-certitude/src/windows.rs > [2]: > https://github.com/Lukasa/rust-certitude/blob/master/rust-certitude/src/osx.rs > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com > _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
