1. Does this affect easy_install? 2. If/because this affects easy_install, should the guidance / suggested package installation tool be [pip]; because pip install_requires backports.ssl_match_hostname
https://pypi.python.org/pypi/backports.ssl_match_hostname On Nov 10, 2015 6:48 PM, "Nick Coghlan" <ncogh...@gmail.com> wrote: > Hi folks, > > I have a confession to make - I dropped the ball on the HTTPS > verification backport proposals in PEP 493, and let the upstream and > downstream approval processes get out of sequence. > > As a result, the RHEL 7.2 beta released back in September incorporates > the HTTPS verification feature backport based on the current PEP 493 > draft, even though that hasn't formally been pronounced as an Active > recommendation by python-dev yet. > > Accordingly, I'm belatedly submitting it for pronouncement now: > https://www.python.org/dev/peps/pep-0493/ > > There's currently no BDFL-Delegate assigned, so if Guido doesn't want > to handle it, we'll need to address that question first. > > Our last discussion back in July seemed to show that folks either > didn't care about the question (because they're using unmodified > upstream versions so the PEP didn't affect them), or else thought the > approach described in the PEP was reasonable, so I'm hoping the > consequences of my mistake won't be too severe. > > Regards, > Nick. > > P.S. I'm aware that this looks like presenting a fait accompli at a > point where it's too late to realistically say "No", but the truth is > that preparation for the Python in Education miniconf at PyCon > Australia ramped up immediately after the July discussion, and then I > personally got confused as to the scope of what was being included in > 7.2 (I mistakenly thought it was just PEP 466 for now, with 476+493 > being deferred to a later release, but it's actually the whole package > of 466+476+493). That's my fault for trying to keep track of too many > things at once (and thus failing at some of them), not anyone else's. > > ================================ > > PEP: 493 > Title: HTTPS verification recommendations for Python 2.7 redistributors > Version: $Revision$ > Last-Modified: $Date$ > Author: Nick Coghlan <ncogh...@gmail.com>, > Robert Kuska <rku...@redhat.com>, > Marc-André Lemburg <m...@lemburg.com> > Status: Draft > Type: Informational > Content-Type: text/x-rst > Created: 10-May-2015 > Post-History: 06-Jul-2015 > > > Abstract > ======== > > PEP 476 updated Python's default handling of HTTPS certificates to be > appropriate for communication over the public internet. The Python 2.7 long > term maintenance series was judged to be in scope for this change, with the > new behaviour introduced in the Python 2.7.9 maintenance release. > > This PEP provides recommendations to downstream redistributors wishing to > provide a smoother migration experience when helping their users to manage > this change in Python's default behaviour. > > > Rationale > ========= > > PEP 476 changed Python's default behaviour to better match the needs and > expectations of developers operating over the public internet, a category > which appears to include most new Python developers. It is the position of > the authors of this PEP that this was a correct decision. > > However, it is also the case that this change *does* cause problems for > infrastructure administrators operating private intranets that rely on > self-signed certificates, or otherwise encounter problems with the new > default > certificate verification settings. > > The long term answer for such environments is to update their internal > certificate management to at least match the standards set by the public > internet, but in the meantime, it is desirable to offer these > administrators > a way to continue receiving maintenance updates to the Python 2.7 series, > without having to gate that on upgrades to their certificate management > infrastructure. > > PEP 476 did attempt to address this question, by covering how to revert the > new settings process wide by monkeypatching the ``ssl`` module to restore > the > old behaviour. Unfortunately, the ``sitecustomize.py`` based technique > proposed > to allow system administrators to disable the feature by default in their > Standard Operating Environment definition has been determined to be > insufficient in at least some cases. The specific case of interest to the > authors of this PEP is the one where a Linux distributor aims to provide > their users with a > `smoother migration path > <https://bugzilla.redhat.com/show_bug.cgi?id=1173041>`__ > than the standard one provided by consuming upstream CPython 2.7 releases > directly, but other potential challenges have also been pointed out with > updating embedded Python runtimes and other user level installations of > Python. > > Rather than allowing a plethora of mutually incompatibile migration > techniques > to bloom, this PEP proposes two alternative approaches that redistributors > may take when addressing these problems. Redistributors may choose to > implement > one, both, or neither of these approaches based on their assessment of the > needs of their particular userbase. > > These designs are being proposed as a recommendation for redistributors, > rather > than as new upstream features, as they are needed purely to support legacy > environments migrating from older versions of Python 2.7. Neither approach > is being proposed as an upstream Python 2.7 feature, nor as a feature in > any > version of Python 3 (whether published directly by the Python Software > Foundation or by a redistributor). > > > Requirements for capability detection > ===================================== > > As these recommendations are intended to cover backports to earlier Python > versions, the Python version number cannot be used as a reliable means for > detecting them. Instead, the recommendations are defined to allow the > presence > or absence of the feature to be determined using the following technique:: > > python -c "import ssl; ssl._relevant_attribute" > > This will fail with `AttributeError` (and hence a non-zero return code) if > the > relevant capability is not available. > > The marker attributes are prefixed with an underscore to indicate the > implementation dependent nature of these capabilities - not all Python > distributions will offer them, only those that are providing a multi-stage > migration process from the legacy HTTPS handling to the new default > behaviour. > > > Recommendation for an environment variable based security downgrade > =================================================================== > > Some redistributors may wish to provide a per-application option to disable > certificate verification in selected applications that run on or embed > CPython > without needing to modify the application itself. > > In these cases, a configuration mechanism is needed that provides: > > * an opt-out model that allows certificate verification to be selectively > turned off for particular applications after upgrading to a version of > Python that verifies certificates by default > * the ability for all users to configure this setting on a per-application > basis, rather than on a per-system, or per-Python-installation basis > > This approach may be used for any redistributor provided version of Python > 2.7, > including those that advertise themselves as providing Python 2.7.9 or > later. > > > Required marker attribute > ------------------------- > > The required marker attribute on the ``ssl`` module when implementing this > recommendation is:: > > _https_verify_envvar = 'PYTHONHTTPSVERIFY' > > This not only makes it straightforward to detect the presence (or absence) > of > the capability, it also makes it possible to programmatically determine the > relevant environment variable name. > > > Recommended modifications to the Python standard library > -------------------------------------------------------- > > The recommended approach to providing a per-application configuration > setting > for HTTPS certificate verification that doesn't require modifications to > the > application itself is to: > > * modify the ``ssl`` module to read the ``PYTHONHTTPSVERIFY`` environment > variable when the module is first imported into a Python process > * set the ``ssl._create_default_https_context`` function to be an alias for > ``ssl._create_unverified_context`` if this environment variable is > present > and set to ``'0'`` > * otherwise, set the ``ssl._create_default_https_context`` function to be > an > alias for ``ssl.create_default_context`` as usual > > > Example implementation > ---------------------- > > :: > > _https_verify_envvar = 'PYTHONHTTPSVERIFY' > > def _get_https_context_factory(): > config_setting = os.environ.get(_https_verify_envvar) > if config_setting == '0': > return _create_unverified_context > return create_default_context > > _create_default_https_context = _get_https_context_factory() > > > Security Considerations > ----------------------- > > Relative to an unmodified version of CPython 2.7.9 or later, this approach > does introduce a new downgrade attack against the default security settings > that potentially allows a sufficiently determined attacker to revert Python > to the vulnerable configuration used in CPython 2.7.8 and earlier releases. > However, such an attack requires the ability to modify the execution > environment of a Python process prior to the import of the ``ssl`` module, > and any attacker with such access would already be able to modify the > behaviour of the underlying OpenSSL implementation. > > > Recommendation for backporting to earlier Python versions > ========================================================= > > Some redistributors, most notably Linux distributions, may choose to > backport > the PEP 476 HTTPS verification changes to modified Python versions based on > earlier Python 2 maintenance releases. In these cases, a configuration > mechanism is needed that provides: > > * an opt-in model that allows the decision to enable HTTPS certificate > verification to be made independently of the decision to upgrade to the > Python version where the feature was first backported > * the ability for system administrators to set the default behaviour of > Python > applications and scripts run directly in the system Python installation > * the ability for the redistributor to consider changing the default > behaviour > of *new* installations at some point in the future without impacting > existing > installations that have been explicitly configured to skip verifying > HTTPS > certificates by default > > This approach should not be used for any Python installation that > advertises > itself as providing Python 2.7.9 or later, as most Python users will have > the > reasonable expectation that all such environments will validate HTTPS > certificates by default. > > > Required marker attribute > ------------------------- > > The required marker attribute on the ``ssl`` module when implementing this > recommendation is:: > > _cert_verification_config = '<path to configuration file>' > > This not only makes it straightforward to detect the presence (or absence) > of > the capability, it also makes it possible to programmatically determine the > relevant configuration file name. > > > Recommended modifications to the Python standard library > -------------------------------------------------------- > > The recommended approach to backporting the PEP 476 modifications to an > earlier > point release is to implement the following changes relative to the default > PEP 476 behaviour implemented in Python 2.7.9+: > > * modify the ``ssl`` module to read a system wide configuration file when > the > module is first imported into a Python process > * define a platform default behaviour (either verifying or not verifying > HTTPS > certificates) to be used if this configuration file is not present > * support selection between the following three modes of operation: > > * ensure HTTPS certificate verification is enabled > * ensure HTTPS certificate verification is disabled > * delegate the decision to the redistributor providing this Python > version > > * set the ``ssl._create_default_https_context`` function to be an alias for > either ``ssl.create_default_context`` or > ``ssl._create_unverified_context`` > based on the given configuration setting. > > > Recommended file location > ------------------------- > > This approach is currently only defined for \*nix system Python > installations. > > The recommended configuration file name is > ``/etc/python/cert-verification.cfg``. > > The ``.cfg`` filename extension is recommended for consistency with the > ``pyvenv.cfg`` used by the ``venv`` module in Python 3's standard library. > > > Recommended file format > ----------------------- > > The configuration file should use a ConfigParser ini-style format with a > single section named ``[https]`` containing one required setting > ``verify``. > > Permitted values for ``verify`` are: > > * ``enable``: ensure HTTPS certificate verification is enabled by default > * ``disable``: ensure HTTPS certificate verification is disabled by default > * ``platform_default``: delegate the decision to the redistributor > providing > this particular Python version > > If the ``[https]`` section or the ``verify`` setting are missing, or if the > ``verify`` setting is set to an unknown value, it should be treated as if > the > configuration file is not present. > > > Example implementation > ---------------------- > > :: > > _cert_verification_config = '/etc/python/cert-verification.cfg' > > def _get_https_context_factory(): > # Check for a system-wide override of the default behaviour > context_factories = { > 'enable': create_default_context, > 'disable': _create_unverified_context, > 'platform_default': _create_unverified_context, # For now :) > } > import ConfigParser > config = ConfigParser.RawConfigParser() > config.read(_cert_verification_config) > try: > verify_mode = config.get('https', 'verify') > except (ConfigParser.NoSectionError, ConfigParser.NoOptionError): > verify_mode = 'platform_default' > default_factory = context_factories.get('platform_default') > return context_factories.get(verify_mode, default_factory) > > _create_default_https_context = _get_https_context_factory() > > > Security Considerations > ----------------------- > > The specific recommendations for the backporting case are designed to work > for > privileged, security sensitive processes, even those being run in the > following > locked down configuration: > > * run from a locked down administrator controlled directory rather than a > normal > user directory (preventing ``sys.path[0]`` based privilege escalation > attacks) > * run using the ``-E`` switch (preventing ``PYTHON*`` environment variable > based > privilege escalation attacks) > * run using the ``-s`` switch (preventing user site directory based > privilege > escalation attacks) > * run using the ``-S`` switch (preventing ``sitecustomize`` based privilege > escalation attacks) > > The intent is that the *only* reason HTTPS verification should be getting > turned off system wide when using this approach is because: > > * an end user is running a redistributor provided version of CPython rather > than running upstream CPython directly > * that redistributor has decided to provide a smoother migration path to > verifying HTTPS certificates by default than that being provided by the > upstream project > * either the redistributor or the local infrastructure administrator has > determined that it is appropriate to override the default upstream > behaviour > (at least for the time being) > > Using an administrator controlled configuration file rather than an > environment > variable has the essential feature of providing a smoother migration path, > even > for applications being run with the ``-E`` switch. > > > Combining the recommendations > ============================= > > If a redistributor chooses to implement both recommendations, then the > environment variable should take precedence over the system-wide > configuration > setting. This allows the setting to be changed for a given user, virtual > environment or application, regardless of the system-wide default > behaviour. > > In this case, if the ``PYTHONHTTPSVERIFY`` environment variable is > defined, and > set to anything *other* than ``'0'``, then HTTPS certificate verification > should be enabled. > > Example implementation > ---------------------- > > :: > > _https_verify_envvar = 'PYTHONHTTPSVERIFY' > _cert_verification_config = '/etc/python/cert-verification.cfg' > > def _get_https_context_factory(): > # Check for am environmental override of the default behaviour > config_setting = os.environ.get(_https_verify_envvar) > if config_setting is not None: > if config_setting == '0': > return _create_unverified_context > return create_default_context > > # Check for a system-wide override of the default behaviour > context_factories = { > 'enable': create_default_context, > 'disable': _create_unverified_context, > 'platform_default': _create_unverified_context, # For now :) > } > import ConfigParser > config = ConfigParser.RawConfigParser() > config.read(_cert_verification_config) > try: > verify_mode = config.get('https', 'verify') > except (ConfigParser.NoSectionError, ConfigParser.NoOptionError): > verify_mode = 'platform_default' > default_factory = context_factories.get('platform_default') > return context_factories.get(verify_mode, default_factory) > > _create_default_https_context = _get_https_context_factory() > > > Copyright > ========= > > This document has been placed into the public domain. > > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com >
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
