[issue23857] Make default HTTPS certificate verification setting configurable

Changes by Nick Coghlan : -- resolution: -> fixed stage: commit review -> resolved status: open -> closed versions: +Python 2.7 -Python 3.5 ___ Python tracker

[issue23857] Make default HTTPS certificate verification setting configurable

Roundup Robot added the comment: New changeset 7a9c9368d79e by Nick Coghlan in branch '2.7': Issue #23857: Implement PEP 493 https://hg.python.org/cpython/rev/7a9c9368d79e -- nosy: +python-dev ___ Python tracker

[issue23857] Make default HTTPS certificate verification setting configurable

Nick Coghlan added the comment: As Robert suggested, I tweaked the envvar tests to be more self-explanatory: https_is_verified = """import ssl, sys; \ status = "Error: _create_default_https_context does not verify certs" \ if

[issue23857] Make default HTTPS certificate verification setting configurable

Robert Kuska added the comment: If test fail it will print out non-telling message which make debugging a little bit hard: FAIL: test__https_verify_envvar (test.test_ssl.ContextTests)

[issue23857] Make default HTTPS certificate verification setting configurable

Nick Coghlan added the comment: Explicitly noting for anyone considering backporting this change (together with PEP 466 & 476) to a long term support release: watch out for https://bugs.python.org/issue22438 The RHEL/CentOS backport includes a reimplementation of sslwrap: *

[issue23857] Make default HTTPS certificate verification setting configurable

Nick Coghlan added the comment: The attached patch adds documentation (to the ssl module docs, the environment variable descriptions and the 2.7 What's New) and also fixes the PYTHONHTTPSVERIFY tests to use a subprocess so they can still run under -E. Assuming the docs pass muster, then the

[issue23857] Make default HTTPS certificate verification setting configurable

Nick Coghlan added the comment: Since PEP 493 is now a standards track PEP, the attached patch provides the reference implementation for the current PEP text. -- Added file: http://bugs.python.org/file41622/pep493_py27_ssl_config.diff ___ Python

[issue23857] Make default HTTPS certificate verification setting configurable

Barry A. Warsaw added the comment: On Oct 14, 2015, at 03:20 PM, Nick Coghlan wrote: >The rationale behind "platform_default" relates to what we put in the default >config file in the RPM. If enable/disable are the only options, then as soon >as the first version ships with "disable" as the

[issue23857] Make default HTTPS certificate verification setting configurable

Nick Coghlan added the comment: The rationale behind "platform_default" relates to what we put in the default config file in the RPM. If enable/disable are the only options, then as soon as the first version ships with "disable" as the default, affected systems will *never* switch to being

[issue23857] Make default HTTPS certificate verification setting configurable

Nick Coghlan added the comment: Yeah, it's the extra 5 years on RHEL 7 that makes me wary. For anything with a shorter life cycle, letting the legacy setting age out likely makes more sense. -- ___ Python tracker

[issue23857] Make default HTTPS certificate verification setting configurable

Barry A. Warsaw added the comment: Re: platform_default - I'm not sure that's a good idea. It hides what's actually happening in some hard to discover place (the code). Probably EIBTI and just go with 'enable' and 'disable'. -- ___ Python tracker

[issue23857] Make default HTTPS certificate verification setting configurable

Nick Coghlan added the comment: First draft of a recommendations PEP: https://hg.python.org/peps/rev/85bc7f13b295 (PEP 493) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23857 ___

[issue23857] Make default HTTPS certificate verification setting configurable

Marc-Andre Lemburg added the comment: Changing the title to reflect that the solution to how to configure Python is still up in the air. I also started a thread on python-dev to get some more feedback. -- title: Make default HTTPS certificate verification setting configurable via

[issue23857] Make default HTTPS certificate verification setting configurable

Changes by Christian Heimes li...@cheimes.de: -- nosy: +christian.heimes ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23857 ___ ___

[issue23857] Make default HTTPS certificate verification setting configurable

Christian Heimes added the comment: Please let me join the party. :) Like Antoine and Donald I'm against an option to disable certificate validation. I truly believe it's the wrong approach for the problem. Users don't *want* to disable security checks either. They disable the check because

[issue23857] Make default HTTPS certificate verification setting configurable

Christian Heimes added the comment: PS: It's also super easy to trust self-signed certificates. All you have to do is to grab the cert and set SSL_CERT_FILE env var: $ openssl s_client -connect host:443 | openssl x509 /path/to/selfsigned.pem $ SSL_CERT_FILE=/path/to/selfsigned.pem python

[issue23857] Make default HTTPS certificate verification setting configurable

Marc-Andre Lemburg added the comment: Those are nice ideas, but you are forgetting two important points: * browsers are typically only being used by single users, applications by potentially hundreds or thousands of users * how should the poor sys admin who's task it is to keep Python

[issue23857] Make default HTTPS certificate verification setting configurable

Nick Coghlan added the comment: Right, the key here is to think like a system administrator, not a developer. Most of those folks are downstream of redistributors (whether commercial ones or community Linux distributions) and relying on one of two things: * tools using the system cert store for

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Nick Coghlan added the comment: After further consideration, I realised there's an important difference between this case and the hash randomisation case: having the -E switch imply hash randomisation was OK, but having it imply HTTPS certificate verification after the system administrator

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Nick Coghlan added the comment: Folks being wary of upgrading to new maintenance releases is already the case - RHEL/CentOS selectively backport things, and other orgs like Google do extensive integration testing before deploying new versions. Folks that only use and write well behaved and

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Marc-Andre Lemburg added the comment: I think this discussion is moving in the wrong direction or least one which won't help people not using some Linux distribution. The use case here is very similar to the hash seed randomization which was also successfully handled using an environment

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Nick Coghlan added the comment: In issue 23955, Steve Dower has suggested introducing config file support for easier control of path configuration when a dedicated Python interpreter runtime is deployed as part of a larger application. If that proposal goes ahead (I think it needs a PEP for

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Nick Coghlan added the comment: I like the idea of a separate 2.7-redistributor branch to capture changes like this, as that would almost *exactly* duplicate the kernel maintenance model. That approach would also mean that we don't have to figure out sensible upstream documentation for

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Robert Kuska added the comment: Le 06/04/2015 13:29, Nick Coghlan a écrit : So while this isn't a feature upstream itself needs, it's one potentially needed by multiple *downstreams*, so in my view it makes sense for us to work with upstream to come up with the one obvious way for

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

R. David Murray added the comment: I believe the original objection was that it made it too easy to globally (and in a not-obvious-to-the-end-user way) disable validation. That argument seems to apply equally well to the proposed patch, so an environment var at least isn't worse; but it does

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Antoine Pitrou added the comment: Environment variables are hidden state. It makes them rather dangerous from a security POV (even more so than a root-modifiable configuration file, since it is less well-defined who can set an environment variable that will be inherited by some process). As

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Nick Coghlan added the comment: The change in 2.7.9 upstream was *absolutely* the right thing for the upstream CPython community to do. The problem was real, it needed to be fixed, and the community fixed it in a way that works just fine for folks in the earlier parts of the technology

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Antoine Pitrou added the comment: Le 06/04/2015 13:29, Nick Coghlan a écrit : So while this isn't a feature upstream itself needs, it's one potentially needed by multiple *downstreams*, so in my view it makes sense for us to work with upstream to come up with the one obvious way for

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Marc-Andre Lemburg added the comment: On 05.04.2015 23:25, R. David Murray wrote: MAL: then what you are arguing for is that the SSL changes in 2.7.9 should not have happened. Which is an argument that Antoine and I at least are sympathetic to :) I think those changes were probably

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

R. David Murray added the comment: Actually I was in favor of an environment variable (or something like that) from the start, because it could be set per-process (making it as close to per-application as we can get from upstream). But a global config file I think is a bad idea (at least in

Re: [issue23857] Make default HTTPS certificate verification setting configurable via global ini file

On 05.04.2015 22:49, Donald Stufft wrote: Donald Stufft added the comment: I don't consider monkey patching a proper way to configure a Python installation. The point is that that TLS validation on/off isn't conceptually a Python level configuration option, that's going to be a per

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Donald Stufft added the comment: I don't consider monkey patching a proper way to configure a Python installation. The point is that that TLS validation on/off isn't conceptually a Python level configuration option, that's going to be a per application configuration option. The monkeypatching

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

R. David Murray added the comment: MAL: then what you are arguing for is that the SSL changes in 2.7.9 should not have happened. Which is an argument that Antoine and I at least are sympathetic to :) -- ___ Python tracker

Re: [issue23857] Make default HTTPS certificate verification setting configurable via global ini file

FWIW: I just ran into a situation where the new approach resulted in pip, setuptools and zc.buildout not working anymore. This was on an AIX system which did come with CA root certificates at all. Now, I knew how to fix this, but the solution was not an obvious one. I had to use truss to figure

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Antoine Pitrou added the comment: This was on a system where Python 2.7.3 had been installed previously. After the upgrade to Python 2.7.9 nothing worked anymore. Who did the upgrade and with which binaries? If you're compiling Python from source, especially for an exotic system, well,

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Nick Coghlan added the comment: PEP 476 *has* a mechanism in it that was supposed to deal with this problem, thus leaving *end users* in full control of the decision on when they upgrade their security infrastructure rather than having that decision arbitrarily imposed on them by a vendor or

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Antoine Pitrou added the comment: By the way, if a vendor wants vendor-specific behaviour, forking the standard library is a normal price to pay. (in this case, the diff wouldn't be large, and it's made against an extremely stable upstream branch) --

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Donald Stufft added the comment: On it's own I think this switch is a bad idea because it's too big of a hammer. Someone shouldn't accidentally disable TLS verification in pip for instance because they wanted to disable TLS verification for some random tool that only hit internal TLS but

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Nick Coghlan added the comment: The discussion isn't on python-ideas yet because I wanted to get a better sense of what might be politically feasible before putting this question to a broader audience. I agree it needs to move there eventually (likely during or after PyCon), and will almost

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Nick Coghlan added the comment: As far as Alex's post goes, it's simply wrong, and I wish he had spoken to me about his frustrations with the significant challenges of infrastructure maintenance in large established organisations before posting it. Red Hat's been fighting the battle for

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Antoine Pitrou added the comment: Le 05/04/2015 12:25, Nick Coghlan a écrit : This does mean spending more time upfront coming up with a way of designing the feature that the core development community considers to be useful independently of backporting considerations (e.g. bringing the

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Donald Stufft added the comment: Now, I knew how to fix this, but the solution was not an obvious one. I had to use truss to figure out where OpenSSL was looking for certificates and the added the Mozilla cert bundle from our egenix-pyopenssl package to make things work again. You also

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Marc-Andre Lemburg added the comment: On 05.04.2015 18:28, Donald Stufft wrote: Donald Stufft added the comment: Now, I knew how to fix this, but the solution was not an obvious one. I had to use truss to figure out where OpenSSL was looking for certificates and the added the Mozilla

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Antoine Pitrou added the comment: Le 05/04/2015 21:26, Marc-Andre Lemburg a écrit : But this is only an example of an application not working anymore because the system's OpenSSL could not verify certificates. In this case, no root CA certs were available. On older systems with proper root

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Marc-Andre Lemburg added the comment: On 05.04.2015 21:36, Antoine Pitrou wrote: And you want the python-dev community to care for that broken situation by bearing the cost of additional maintenance and security risk in implementing the new configuration options? No, I want to be able to

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Donald Stufft added the comment: No, I want to be able to easily disable the newly added checks in 2.7.9+ to get systems such as these behave the same as with 2.7.8, since without this option, people using these system are going to be forced to stick with buggy 2.7.8 systems. Why is the

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Marc-Andre Lemburg added the comment: On 05.04.2015 21:48, Donald Stufft wrote: Donald Stufft added the comment: No, I want to be able to easily disable the newly added checks in 2.7.9+ to get systems such as these behave the same as with 2.7.8, since without this option, people using

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

R. David Murray added the comment: Really these arguments make it sound like 2.7.9 never should have happened. Given that it did, Nick has not addressed the question of why the vendors maintaining this simple patch (given that it addresses what he sees as their need) is not a viable option.

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Antoine Pitrou added the comment: I agree with Donald on all points. This shouldn't be done at the language level at all (why should it apply only to Python-written tools?). Having a centralized setting saying I relinquish security on HTTPS accesses sounds like a bad idea. And if this is

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Donald Stufft added the comment: I'd really rather not add this to Python itself. If downstream wants to patch their Pythons to do it that is their prerogative. There's some legacy at play here of course, however I don't think that Python upstream is the right place to deal with that. One

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

R. David Murray added the comment: Changing the title to be specific to the proposed patch. -- title: [RFE] Make default HTTPS certificate verification setting configurable - Make default HTTPS certificate verification setting configurable via global ini file

[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

Barry A. Warsaw added the comment: +1 for keyed by site There have been a number of issues over the years for which a configuration file (or files) would have been useful. I think a discussion over on python-ideas is the right way to move forward on this point. --