On Mar 23, 2014, at 3:07 AM, Nick Coghlan <ncogh...@gmail.com> wrote:
> Several significant changes in this revision: > > - scope narrowed to just Python 2.7 plus permission for commercial > redistributors to use the same strategy in their long term support > releases > - far more explicit that this is about inviting potential corporate > contributors to address the situation for the benefit of the overall > Python ecosystem, not offering to fix it for them for free > - clarified that third party integration testing services would need > to be updated to support testing against multiple Python 2.7 minor > releases For what it’s worth, I have an outstanding PR against Travis CI that would make this trivial for them at least. They are a pretty popular CI service especially for OSS projects. I made that PR for unrelated reasons but it could at least serve as a template for other projects to do the same thing. > - explicit sections on why I don't think the status quo is > sustainable, why I don't think Python 2.8 would actually solve the > problem, and why I think a PyPI based solution not only wouldn't solve > the problem, but would be rather difficult to get working in the first > place > - be completely explicit that I am *not* speaking on behalf of Red Hat > at this point and have no authority to make commitments on their > behalf. Instead, I'm looking for upstream consensus that 1) this is a > genuine problem that needs to be solved; 2) we're open to corporate > assistance in solving it; and 3) we have a pretty good idea what help > we actually want. If all that happens, *then* I can take up the issue > internally to try to get us some help in maintaining the proposed > solution (hopefully other folks with corporate influence can do the > same, and we may actually get some ongoing assistance with upstream > maintenance out of this, rather than having our downstream > redistributors continue to take us for granted). > > Diff: http://hg.python.org/peps/rev/2e82209dda21 > Updated web version: http://www.python.org/dev/peps/pep-0466/ > > Advance warning: while I was able to get this revision turned around > pretty quickly, future revisions are likely to take a fair bit longer. > It was already a rather busy month before I decided to start this > discussion on top of everything else :) > > Cheers, > Nick. > > > PEP: 466 > Title: Network Security Enhancement Exception for Python 2.7 > Version: $Revision$ > Last-Modified: $Date$ > Author: Nick Coghlan <ncogh...@gmail.com>, > Status: Draft > Type: Informational > Content-Type: text/x-rst > Created: 23-Mar-2014 > Post-History: 23-Mar-2014 > > > Abstract > ======== > > Most CPython tracker issues are classified as errors in behaviour or > proposed enhancements. Most patches to fix behavioural errors are > applied to all active maintenance branches. Enhancement patches are > restricted to the default branch that becomes the next Python version. > > This cadence works reasonably well during Python's normal 18-24 month > feature release cycle, which is still applicable to the Python 3 series. > However, the age of the standard library in Python 2 has now reached a point > where it is sufficiently far behind the state of the art in network security > protocols for it to be causing real problems in commercial use cases > where upgrading to Python 3 in the near term may not be practical. > > Accordingly, this PEP relaxes the normal restrictions by allowing > enhancements to be applied in Python 2.7 maintenance releases for standard > library components that have implications for the overall security of the > internet. In particular, the exception will apply to: > > * the ``ssl`` module > * the ``hashlib`` module > * the ``hmac`` module > * the ``sha`` module (Python 2 only) > * the components of other networking modules that make use of these modules > * the components of the ``random`` and ``os`` modules that are relevant to > cryptographic applications > * the version of OpenSSL bundled with the binary installers > > Proposed backports for these modules will still need to undergo normal > backwards compatibility assessments, but new features will be permitted where > appropriate, making it easier to implement secure networked software in > Python, even for software that needs to remain compatible with older feature > releases of Python. > > While this PEP does not make any changes to the core development team's > handling of security-fix-only branches that are no longer in active > maintenance, it *does* recommend that commercial redistributors providing > extended support periods for the Python standard library either adopt a > similar approach to ensuring that the secure networking infrastructure > keeps pace with the evolution of the internet, or else disclaim support > for the use of older versions in roles that involving connecting > directly to the public internet. > > > Exemption Policy > ================ > > Under this policy, the following network security related modules are > granted a blanket exemption to the restriction against adding new features > in maintenance releases, for the purpose of keeping their APIs aligned with > their counterparts in the latest feature release of Python 3: > > * the ``ssl`` module > * the ``hashlib`` module > * the ``hmac`` module > * the ``sha`` module (Python 2 only) > > This exemption applies to *all* proposals to backport backwards compatible > changes in these modules to Python 2.7 maintenance releases. This choice is > made deliberately to ensure that the "feature or fix?" argument isn't simply > replaced by a "security related or not?" argument. These particular modules > are inherently security related, and all enhancements to them improve > Python's capabilities as a platform for development of secure networked > software. > > As part of this policy, permission is also granted to upgrade to newer > feature releases of OpenSSL when preparing the binary installers > for new maintenance releases of CPython. > > In addition to the above blanket exemption, a conditional exemption is > granted for these modules that may include some network security related > features: > > * the ``os`` module (specifically ``os.urandom``) > * the ``random`` module > * networking related modules that depend on one or more of the network > security related modules listed above > > This more limited exemption for these modules requires that the *specific* > enhancement being proposed for backporting needs to be justified as being > network security related. If the enhancement under discussion is designed > to take advantage of a new feature in one of the network security related > modules, then that will be taken as implying that the enhancement is > security related. > > > Backwards Compatibility Considerations > ====================================== > > This PEP does *not* grant any general exemptions to the usual backwards > compatibility policy for maintenance releases. Instead, by explicitly > encouraging the use of feature based checks and explicitly opting in to > less secure configurations, it is designed to make it easier to provide > more "secure by default" behaviour in future feature releases, while still > limiting the risk of breaking currently working software when upgrading to > a new maintenance release. > > In *all* cases where this policy is applied to backport enhancements to > maintenance releases, it MUST be possible to write cross-version compatible > code that operates by "feature detection" (for example, checking for > particular attributes in the module), without needing to explicitly check > the Python version. > > It is then up to library and framework code to provide an appropriate warning > and fallback behaviour if a desired feature is found to be missing. While > some especially security sensitive software MAY fail outright if a desired > security feature is unavailable, most software SHOULD instead continue > operating using a slightly degraded security configuration. > > Affected APIs SHOULD be designed to allow library and application code to > perform the following actions after detecting the presence of a relevant > network security related feature: > > * explicitly opt in to more secure settings (to allow the use of enhanced > security features in older maintenance releases of Python) > * explicitly opt in to less secure settings (to allow the use of newer Python > feature releases in lower security environments) > * determine the default setting for the feature (this MAY require explicit > Python version checks to determine the Python feature release, but MUST > NOT require checking for a specific maintenance release) > > Security related changes to other modules (such as data format processing > libraries) will continue to be made available as backports and new modules > on the Python Package Index, as independent distribution remains the > preferred approach to handling software that needs to evolve faster than > the standard library. Refer to the `Motivation and Rationale`_ section for > a review of the characteristics that make the secure networking > infrastructure worthy of special consideration. > > > Other Considerations > ==================== > > Maintainability > --------------- > > This policy does NOT represent a commitment by volunteer contributors to > actually backport network security related changes from the Python 3 series > to the Python 2 series. Rather, it is intended to send a clear signal to > potential corporate contributors that the core development team are willing > to review and merge corporate contributions that put this policy into > effect. > > Backporting security related fixes and enhancements to earlier versions is > a common service for commercial redistributors to offer to their customers. > This policy represents an explicit invitation to contribute some of those > changes back to the upstream community in cases where they are likely to > have a broad impact that helps improve the security of the internet as a > whole. > > > Documentation > ------------- > > All modules that take advantage of this policy to backport network > security related enhancements to earlier Python versions MUST include > a "Security Considerations" section in their documentation. > > In addition to any other module specific contents, this section MUST > enumerate key security enhancements and fixes (with CVE identifiers where > applicable), along with the feature and maintenance releases that first > included them. > > > Security releases > ----------------- > > This PEP does not propose any changes to the handling of security > releases - those will continue to be source only releases that > include only critical security fixes. > > However, the recommendations for library and application developers are > deliberately designed to accommodate commercial redistributors applying > this policy to any Python release series that is either in security > fix only mode, or has been declared "end of life" by the core development > team. > > Whether or not redistributors choose to exercise that option will be up > to the redistributor. > > > Integration testing > ------------------- > > Third party integration testing services would likely need to start > offering users a choice of multiple Python 2.7.x versions to test against, > to ensure that the application is correctly degrading gracefully if it > attempts to use newer networking features on maintenance releases that > are too old to provide them. > > > Evolution of this Policy > ======================== > > The key requirement for a module to be considered for inclusion in this > policy (whether under a blanket or conditional exemption) is that it must > have security implications *beyond* the specific application that is written > in Python and the system that application is running on. Thus the focus on > network security protocols and related cryptographic infrastructure - Python > is a popular choice for the development of web services and clients, and > thus the capabilities of widely used Python versions have implications for > the security design of other services that may be using newer versions of > Python or other development languages. > > The intent behind this requirement is to minimise any impact that the > introduction of this policy may have on the stability and compatibility of > maintenance releases. It would be thoroughly counterproductive if end > users became as cautious about updating to new Python maintenance releases > as they are about updating to new feature releases. > > > Motivation and Rationale > ======================== > > This PEP can be seen as a more targeted version of the "faster standard > library release cycle" proposals discussed in PEP 407 and PEP 413, > focusing specifically on those areas which have implications beyond the > Python community. > > > Background > ---------- > > The creation of this PEP was prompted primarily by the aging SSL support in > the Python 2 series. As of March 2014, the Python 2.7 SSL module is > approaching four years of age, and the SSL support in the still popular > Python 2.6 release had its feature set locked six years ago. > > These are simply too old to provide a foundation that can be recommended > in good conscience for secure networking software that operates over the > public internet, especially in an era where it is becoming quite clearly > evident that advanced persistent security threats are even more widespread > and more indiscriminate in their targeting than had previously been > understood. While they represented reasonable security infrastructure in > their time, the state of the art has moved on, and we need to investigate > mechanisms for effectively providing more up to date network security > infrastructure for users that, for whatever reason, are not currently in > a position to migrate to Python 3. > > While the use of the system OpenSSL installation addresses many of these > concerns on Linux platforms, it doesn't address all of them, and in the > case of the binary installers for Windows and Mac OS X that are published > on python.org, the version of OpenSSL used is entirely within the control > of the Python core development team, and currently limited to OpenSSL > maintenance releases for the version initially shipped with the corresponding > Python feature release. > > With increased popularity comes increased responsibility, and this policy > aims to acknowledge the fact that Python's popularity and adoption has now > reached a level where some of our design and policy decisions have > significant implications beyond the Python development community. > > As one example, the Python 2 ``ssl`` module does not support the Server > Name Identification standard. While it is possible to obtain SNI support > by using the third party ``requests`` client library, actually doing so > currently requires using not only ``requests`` and its embedded dependencies, > but also half a dozen or more additional libraries. The lack of support > in the Python 2 series thus serves as an impediment to making effective > use of SNI on servers, as Python 2 clients will frequently fail to handle > it correctly. > > Another more critical example is the lack of SSL hostname matching in the > Python 2 standard library - it is currently necessary to rely on a third > party library, such as ``requests`` or ``backports.ssl_match_hostname`` to > obtain that functionality in Python 2. > > The Python 2 series also remains more vulnerable to remote timing attacks > on security sensitive comparisons than the Python 3 series, as it lacks a > standard library equivalent to the timing attack resistant > ``hmac.compare_digest()`` function. While appropriate secure comparison > functions can be implemented in third party extensions, may users don't > even consider the problem and use ordinary equality comparisons instead > - while a standard library solution doesn't automatically fix that problem, > it *does* make the barrier to resolution much lower once the problem is > pointed out. > > My position on the ongoing transition from Python 2 to Python 3 has long > been that Python 2 remains a supported platform for the core development > team, and that commercial support will remain available well after upstream > maintenance ends. However, in the absence of this network security > enhancement policy, that position is difficult to justify when it comes to > software that operates over the public internet. Just as many developers > consider it too difficult to develop truly secure modern networked software > in C/C++ (largely due to the challenges associated with manual > memory management), I anticipate that in the not too distant future, it > will be considered too difficult to develop truly secure modern networked > software using the Python 2 series (some developers would argue that we > have already reached that point). > > > Alternative: advise developers of networked software to migrate to Python 3 > --------------------------------------------------------------------------- > > This alternative represents the status quo. Unfortunately, it has proven > to be unworkable in practice, as the backwards compatibility implications > mean that this is a non-trivial migration process for large applications > and integration projects. > > Now that we're fully aware of the impact the limitations in Python 2 may be > having on the evolution of internet security standards, I no longer believe > that it is reasonable to expect platform and application developers to > resolve all of the latent defects in an application's Unicode correctness > solely in order to gain access to the network security enhancements > available in Python 3. > > While (as far as I am aware) Ubuntu has successfully switched to Python 3.4 > as its main Python interpreter for its 14.04 LTS release, Fedora still > has a lot of work to do to migrate, and it will take a non-trivial amount > of time to migrate the relevant infrastructure components. While Red Hat > are also actively working to make it easier for users to use more recent > versions of Python on our stable platforms, it's going to take time for > those efforts to start having an impact on end users' choice of version, > and those changes won't affect the core tools regardless. > > The OpenStack migration to Python 3 is also still in its infancy, and even > though that's a project with an extensive and relatively robust automated > test suite, it's large enough that it is going to take quite some time > to migrate. > > And that's just three of the highest profile open source projects that > make heavy use of Python. Given the likely existence of large amounts of > legacy code that lacks the kind of automated regression test suite needed > to help support a migration from Python 2 to Python 3. The key point of > this PEP is that those situations affect more people than just the > developers and users of the affected application: their existence becomes > something that developers of secure networked services need to take into > account as part of their security design. > > As Terry Reedy noted, if we try to persist with the status quo, the likely > outcome is that commercial redistributors will attempt to do something > like this on behalf of their customers *anyway*, but in a potentially > inconsistent and ad hoc manner. By drawing the scope definition process > into the upstream project we are in a better position to influence the > approach taken to address the situation and to help ensure some consistency > across redistributors. > > The problem is real, so *something* needs to change, and this PEP describes > my currently preferred approach to addressing the situation. > > > Alternative: create and release Python 2.8 > ------------------------------------------ > > With sufficient corporate support, it likely *would* be possible to create > and release Python 2.8 (it's highly unlikely such a project would garner > enough interest to be achievable with only volunteers). However, this > wouldn't actually solve the problem, as the aim is to provide a *relatively > low impact* way to incorporate enhanced security features into integrated > products and deployments that make use of Python 2. Upgrading to a new > Python feature release would mean both more work for the core development > team, as well as a more disruptive update that most potential end users > would likely just skip entirely. > > Attempting to create a Python 2.8 release would also bring in suggestions > to backport many additional features from Python 3 (such as ``tracemalloc`` > and the improved coroutine support). > > This is not a recommended approach, as it would involve substantial > additional work for a result that is actually less effective as a solution > to the original problem (the widespread use of the aging network security > infrastructure in Python 2). > > > Alternative: distribute the security enhancements via PyPI > ---------------------------------------------------------- > > While it initially appears to be an attractive and easier to manage > approach, there are actually several significant problems with this > idea. > > Firstly, this PEP encompasses a non-trivial portion of the standard library. > It's not just the underlying SSL support, but also the libraries for other > network protocols like HTTP, FTP, IMAP, and POP3 that integrate with the > SSL infrastructure to provide secure links, and that's just the protocols > in the standard library. Even if an API compatible ``ssl2`` module was > made available, it would need to be imported and injected > into ``sys.modules`` as ``ssl`` before importing any other module that > needed it. > > Secondly, this is complex, low level, cross-platform code that integrates > with the underlying operating system across a variety of POSIX platforms > (including Mac OS X) and Windows. The CPython BuildBot fleet is already set > up to handle continuous integration in that context, but most of the > freely available continuous integration services just offer Linux, and > perhaps paid access to Windows. Those services work reasonably well for > software that largely runs on the abstraction layers offered by Python and > other dynamic languages, but won't suffice for the kind of code involved > here. > > The OpenSSL dependency for the network security support also qualifies as > the kind of "complex binary dependency" that isn't yet handled well by the > ``pip`` based software distribution ecosystem. Relying on a binary > dependency also creates potential compatibility problems for ``pip`` when > running on other interpreters like ``PyPy``. > > Another practical problem with the idea is the fact that ``pip`` itself > relies on the ``ssl`` support in the standard library (with some additional > support from a bundled copy of ``requests``, which in turn bundles > ``backport.ssl_match_hostname``), and hence would require any replacement > module to also be bundled within ``pip``. This wouldn't pose any > insurmountable difficulties (it's just another dependency to vendor), but > it *would* mean yet another copy of OpenSSL to keep up to > date. > > This approach also has the same flaw as all other "improve security by > renaming things" approaches: they completely miss the users who most need > help, and raise significant barriers against being able to encourage users > to do the right thing when their infrastructure supports it (since > "use this other module" is a much higher impact change than "turn on this > higher security setting"). Deprecating the aging SSL infrastructure in the > standard library in favour of an external module would be even more user > hostile than taking the risk of trying to upgrade it in place. > > Last, but certainly not least, this approach suffers from the same problem > as the idea of doing a Python 2.8 release: likely not solving the actual > problem. Commercial redistributors of Python are set up to redistribute > *Python*, and a pre-existing set of additional packages. Getting new > packages added to the pre-existing set *can* be done, but means going > around to each and every redistributor and asking them to update their > repackaging process accordingly. By contrast, the approach described in > this PEP would require redistributors to *opt out* of the security > enhancements, which most of them are unlikely to do. > > > Open Questions > ============== > > * What are the risks associated with allowing OpenSSL to be updated to > new feature versions in the Windows and Mac OS X binary installers for > maintenance releases? Currently we just upgrade to the appropriate > OpenSSL maintenance releases, rather than switching to the latest > feature release. In particular, is it possible Windows C extensions may > be linking against the Python provided OpenSSL module? > > * Are there any other security relevant modules that should be covered > by either a blanket or conditional exemption? > > > Disclosure of Interest > ====================== > > The author of this PEP currently works for Red Hat on test automation tools. > If this proposal is accepted, I will be strongly encouraging Red Hat to take > advantage of the resulting opportunity to help improve the overall security > of the Python ecosystem. However, I do not speak for Red Hat in this matter, > and cannot make any commitments on Red Hat's behalf. > > > Acknowledgements > ================ > > Thanks to Christian Heimes for his recent efforts on greatly improving > Python's SSL support in the Python 3 series, and a variety of members of > the Python community for helping me to better understand the implications > of the default settings we provide in our SSL modules, and the impact that > tolerating the use of SSL infrastructure that was defined in 2010 > (Python 2.7) or even 2008 (Python 2.6) potentially has for the security > of the web as a whole. > > Christian and Donald Stufft also provided valuable feedback on a preliminary > draft of this proposal. > > Thanks also to participants in the python-dev mailing list thread [1]_ > > > References > ========== > > .. [1] https://mail.python.org/pipermail/python-dev/2014-March/133334.html > > > Copyright > ========= > > This document has been placed in the public domain. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/donald%40stufft.io ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
