[Python-Dev] 3.4 buildbots available
Hello, I've created the 3.4 category in the buildbots setup: http://buildbot.python.org/all/waterfall?category=3.4.stable I've also retired 3.2 and 3.3 buildbots. Someone will have to update the text and URLs at https://www.python.org/dev/buildbot/. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.4 buildbots available
On Sat, Mar 22, 2014, at 11:10, Antoine Pitrou wrote: Hello, I've created the 3.4 category in the buildbots setup: http://buildbot.python.org/all/waterfall?category=3.4.stable I've also retired 3.2 and 3.3 buildbots. Someone will have to update the text and URLs at https://www.python.org/dev/buildbot/. We probably want a http://buildbot.python.org/3.4.stable/ redirect, too, then. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Folks, I have just posted a proposal to change the way we treat enhancements that relate to Python's support for network security enhancements. I now see these as categorically different from most other enhancements, as they have implications for the evolution of internet security as a whole, rather than being limited to affecting the security of individual applications written in Python. It is similar to the previous IDLE policy exception PEP, where we decided that cross version consistency of IDLE superseded the general policy against backporting enhancements to maintenance branches. This new PEP proposes a similar exception for network security related changes, such that the *capabilities* of all still maintained versions should be kept equivalent, with only the *default settings* varying across up to date feature releases. The policy is also designed to make it easier for us to ratchet up the default security settings in future feature releases. However, it's more complex than the IDLE PEP (since it's more invasive), and there are still a few open questions listed. This topic involves a complex balance between encouraging and supporting good security practices and limiting the risk of failures for users upgrading to new maintenance releases, so I'd ask that folks take time to read and consider the implications of the full PEP in the broader context of today's internet before posting comments on specific details, or indicating a preference one way or the other in terms of the overall proposal. Full PEP included inline below, and available in more readable form at http://www.python.org/dev/peps/pep-0466/ Regards, Nick. PEP: 466 Title: Network Security Enhancement Exception for All Branches Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan ncogh...@gmail.com, Status: Draft Type: Informational Content-Type: text/x-rst Created: 23-Mar-2014 Post-History: 23-Mar-2014 Abstract Most CPython tracker issues are classified as errors in behaviour or proposed enhancements. Most patches to fix behavioural errors are applied to all active maintenance branches. Enhancement patches are restricted to the default branch that becomes the next Python version. This PEP relaxes the latter restriction allowing enhancements to be applied to maintenance branches for standard library components that have implications for the overall security of the internet. In particular, the exception will apply to: * the ``ssl`` module * the ``hashlib`` module * the ``hmac`` module * the ``sha`` module (Python 2 only) * the components of other networking modules that make use of these modules * the components of the ``random`` and ``os`` modules that are relevant to cryptographic applications * the version of OpenSSL bundled with the binary installers Changes to these modules will still need to undergo normal backwards compatibility assessments, but new features will be permitted where appropriate, making it easier to implement secure networked software in Python, even for software that needs to remain compatible with older feature releases of Python. Exemption Policy Under this policy, the following network security related modules are granted a blanket exemption to the restriction against adding new features in maintenance releases: * the ``ssl`` module * the ``hashlib`` module * the ``hmac`` module * the ``sha`` module (Python 2 only) This exemption applies to *all* proposals to backport backwards compatible features in these modules to active maintenance branches. This choice is made deliberately to ensure that the feature or fix? argument isn't simply replaced by a security related or not? argument. These particular modules are inherently security related, and all enhancements to them improve Python's capabilities as a platform for development of secure networked software. As part of this policy, permission is also granted to upgrade to newer feature releases of OpenSSL when preparing the binary installers for new maintenance releases of CPython. In addition to the above blanket exemption, a conditional exemption is granted for these modules that may include some network security related features: * the ``os`` module (primarily ``os.urandom``) * the ``random`` module * networking related modules that depend on one or more of the network security related modules listed above This more limited exemption for these modules requires that the *specific* enhancement being proposed for backporting needs to be justified as being network security related. If the enhancement under discussion is designed to take advantage of a new feature in one of the network security related modules, then that will be taken as implying that the enhancement is security related. Backwards Compatibility Considerations == This PEP does *not* grant any general exemptions to the usual backwards compatibility policy for maintenance releases. Instead, it is designed to
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Thanks for putting this together Nick. I suspect it goes without saying that I'm wildly +1 on this as a whole. I'm in favor of leaving it somewhat implicit as to exactly which networking modules concern the health of the internet as a whole. Alex ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sat, Mar 22, 2014, at 14:11, Nick Coghlan wrote: Folks, I have just posted a proposal to change the way we treat enhancements that relate to Python's support for network security enhancements. I think the PEP should also address security-mode releases. Do the same exceptions apply? Does anyone really want to backport features to Python 3.1? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 22 March 2014 at 21:11:24, Nick Coghlan (ncogh...@gmail.com(mailto:ncogh...@gmail.com)) wrote: Folks, I have just posted a proposal to change the way we treat enhancements that relate to Python's support for network security enhancements. I now see these as categorically different from most other enhancements, as they have implications for the evolution of internet security as a whole, rather than being limited to affecting the security of individual applications written in Python. I am 100%, overwhelmingly in favour of this. Without this PEP, Python 2.7 is a security liability, any it becomes nothing short of irresponsible to use Python 2.7 for any form of networking code that hits the open internet. On top of that, the current state of the ssl module means that Python 2.7 and earlier cannot ever support a standard-compliant implementation of things like HTTP/2. That’s a fairly tragic state of affairs for 2.7, especially if we’re supposed to claim with a straight face that it’s acceptable to still use Python 2.7. Treat this as my strong +1. Additionally, I’m happy to volunteer my time and limited expertise to help make this happen. I’ll help work on back porting things, review code, write docs: whatever it takes to get this to happen. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/22/2014 05:11 PM, Nick Coghlan wrote: In addition to any other module specific contents, this section MUST enumerate key security enhancements and fixes (with CVE identifiers where applicable), and the Incomplete sentence. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlMuBosACgkQ+gerLs4ltQ74ngCfS7MLZHtVfS7f6x+9mnOIsp+c +CUAoIc5rFIuNJvMephnlAuPhkPXZspy =N8Xk -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
I think the pep doesn't mandate that someone does. It still requires someone to care enough to actually write the patch. It just allows such a patch to be merged. On Mar 22, 2014, at 5:32 PM, Benjamin Peterson benja...@python.org wrote: Does anyone really want to backport features to Python 3.1? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Am 22.03.14 22:17, schrieb Cory Benfield: I am 100%, overwhelmingly in favour of this. Without this PEP, Python 2.7 is a security liability, any it becomes nothing short of irresponsible to use Python 2.7 for any form of networking code that hits the open internet. Agreed (although this might be a slight overstatement if taken literally). However, the right consequence should be to use Python 3.4 instead. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Just use Python 3.4 ignores the reality of production software. I wish it were that simple because I love 3.4 On Mar 22, 2014, at 6:16 PM, Martin v. Löwis mar...@v.loewis.de wrote: Am 22.03.14 22:17, schrieb Cory Benfield: I am 100%, overwhelmingly in favour of this. Without this PEP, Python 2.7 is a security liability, any it becomes nothing short of irresponsible to use Python 2.7 for any form of networking code that hits the open internet. Agreed (although this might be a slight overstatement if taken literally). However, the right consequence should be to use Python 3.4 instead. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Am 22.03.14 22:11, schrieb Nick Coghlan: Title: Network Security Enhancement Exception for All Branches I'm -0 on the general idea, and -1 on the inclusion of the 2.7 branch into the policy. The PEP trades security concerns against stability and maintainability. I see a maintenance threat coming out of it: existing installations will fail to operate after updating Python and some support libraries. Bug fix releases ought to focus on maintenance much more than moving along with general security threats. Software that ages needs to be retired, instead of trying to fix it endlessly (which is a futile exerciese, anyway). Everybody knows it, and has accepted it, so why should we fight what seems to be a fundamental law of software evolution? In the specific case of 2.7, I am hoping that we retire the branch soon. If so, it would be counter-productive to backport the current features to 2.7. They are still in progress, and users wouldn't be helped with getting a new feature in just the final bug fix release of 2.7, with no chance of then getting updates of the same feature, as there won't be any further bugfix releases. Finally, doing this in the 2.7 branch likely involves more effort than I'm personally willing to provide. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Am 22.03.14 23:05, schrieb Donald Stufft: I think the pep doesn't mandate that someone does. It still requires someone to care enough to actually write the patch. It just allows such a patch to be merged. Does it actually? Unfortunately, I never got around to writing the PEP on security-only branches, so the terminology of PEP 466 is unclear. It says maintenance brances, which could mean three things: - all branches that have ever been made off default in hg.python.org/cpython (i.e. back to 2.0). This would be the traditional definition of maintenance branch in a version control system, but is probably not intended by the PEP - all branches from which still releases are made (whether security-only or bug fix) - all bug fix branches Clarification of this is just a minor nit, though - although the maintainers of the old maintenance branches should (IMO) have a say whether they accept a certain change on a certain branch. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 23 March 2014 08:14, Martin v. Löwis mar...@v.loewis.de wrote: Am 22.03.14 22:11, schrieb Nick Coghlan: Finally, doing this in the 2.7 branch likely involves more effort than I'm personally willing to provide. Believe me, I'll be escalating these concerns at work this week. We have to shoulder a lot of the responsibility for creating this problem in the first place, so I think it's fair for people to expect us to help fix it. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Am 22.03.14 23:21, schrieb Donald Stufft: Just use Python 3.4 ignores the reality of production software. I wish it were that simple because I love 3.4 But I think so does the PEP (i.e. ignore the reality of production software). The risk of breaking existing installations (which might not be affected by the purported security threat being fixed) is just too high. Users that feel threatened by the lack of SSL features in Python should have long started planning on how to get to Python 3. It's not that they haven't been warned. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 23 March 2014 08:23, Martin v. Löwis mar...@v.loewis.de wrote: Am 22.03.14 23:05, schrieb Donald Stufft: I think the pep doesn't mandate that someone does. It still requires someone to care enough to actually write the patch. It just allows such a patch to be merged. Does it actually? Unfortunately, I never got around to writing the PEP on security-only branches, so the terminology of PEP 466 is unclear. It says maintenance brances, which could mean three things: - all branches that have ever been made off default in hg.python.org/cpython (i.e. back to 2.0). This would be the traditional definition of maintenance branch in a version control system, but is probably not intended by the PEP - all branches from which still releases are made (whether security-only or bug fix) - all bug fix branches Clarification of this is just a minor nit, though - although the maintainers of the old maintenance branches should (IMO) have a say whether they accept a certain change on a certain branch. Both good points, I'll clarify them in the PEP. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
In article cadisq7czsp1flv31izz01_9avgyzsc1j6+d2t5aup2byu97...@mail.gmail.com, Nick Coghlan ncogh...@gmail.com wrote: I have just posted a proposal to change the way we treat enhancements that relate to Python's support for network security enhancements. +1 [...] Open Questions == * What are the risks associated with allowing OpenSSL to be updated to new feature versions in the Windows and Mac OS X binary installers for maintenance releases? Regarding the python.org binary installers, I think past practice has been for us to update third-party libraries as necessary in maintenance releases when there is good cause and with the concurrence of the release manager, so I don't see this as a big issue. For the OS X binary installer, the issue for OpenSSL has been that we dynamically link to the system-supplied OpenSSL libraries and that, for various reasons, Apple has deprecated (and frozen at non-current OpenSSL releases) the use of those libraries in favor of their own security frameworks. So, for multiple reasons, including the risk that OpenSSL may be dropped from an upcoming major release of OS X, we need to start supplying our own version with all OS X binary installers. That's the plan regardless of the outcome of this PEP. -- Ned Deily, n...@acm.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 23 March 2014 08:16, Martin v. Löwis mar...@v.loewis.de wrote: Am 22.03.14 22:17, schrieb Cory Benfield: I am 100%, overwhelmingly in favour of this. Without this PEP, Python 2.7 is a security liability, any it becomes nothing short of irresponsible to use Python 2.7 for any form of networking code that hits the open internet. Agreed (although this might be a slight overstatement if taken literally). However, the right consequence should be to use Python 3.4 instead. The PEP does try to address this point in the Motivation Rationale sections: = Requiring that latent defects in an application's Unicode correctness be addressed in order to migrate to Python 3 is not a reasonable alternative recommendation, especially given the likely existence of legacy code that lacks the kind of automated regression test suite needed to help support a migration from Python 2 to Python 3. The key point of this PEP is that those situations affect more people than just the developers and users of the affected application: their existence becomes something that developers of secure networked services need to take into account as part of their security design. By making it more feasible to enhance the security of the Python 2 series, we can help contribute to the evolution of a more secure internet for all concerned. = Hard to maintain legacy software is a fact of life, and way too much of it is exposed to the public internet. This PEP is about doing what we can to mitigate the damage caused both by other people's mistakes, and also the inherent challenges of migrating from the error prone POSIX text model to something more reasonable. I *don't* think its reasonable to expect us to do this without support from the corporate users that caused the problem in the first place (by continuing to deploy older versions of Python without investing adequately in their upkeep), so I'd encourage everyone employed by a commercial user of Python to remind their management chains of the risks of failing to invest development time in any upstream dependencies that they expect to keep pace with the dynamic nature of the internet. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Am 22.03.14 23:33, schrieb Nick Coghlan: Hard to maintain legacy software is a fact of life, and way too much of it is exposed to the public internet. This PEP is about doing what we can to mitigate the damage caused both by other people's mistakes, and also the inherent challenges of migrating from the error prone POSIX text model to something more reasonable. I *don't* think its reasonable to expect us to do this without support from the corporate users that caused the problem in the first place (by continuing to deploy older versions of Python without investing adequately in their upkeep), so I'd encourage everyone employed by a commercial user of Python to remind their management chains of the risks of failing to invest development time in any upstream dependencies that they expect to keep pace with the dynamic nature of the internet. I hope indeed you are successful in activating resources. However, putting them on this backporting project seems like a waste. They should rather go into porting stuff to 3.x where people need it. As responsible maintainers, we should just advise our users that Python 2.7 is a dead horse, and that they should stop riding it. More professionally, we should set an official end-of-life date for 2.7 (alas, we should have done that two years ago). I hope that the language summit can agree to stopping bug fix releases for 2.7 in 2014. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 22.03.2014 22:11, Nick Coghlan wrote: PEP: 466 Title: Network Security Enhancement Exception for All Branches +1 -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 22 2014) Python Projects, Consulting and Support ... http://www.egenix.com/ mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2014-03-29: PythonCamp 2014, Cologne, Germany ... 7 days to go 2014-04-09: PyCon 2014, Montreal, Canada ... 18 days to go 2014-04-29: Python Meeting Duesseldorf ... 38 days to go eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 23 March 2014 08:40, Martin v. Löwis mar...@v.loewis.de wrote: Am 22.03.14 23:33, schrieb Nick Coghlan: Hard to maintain legacy software is a fact of life, and way too much of it is exposed to the public internet. This PEP is about doing what we can to mitigate the damage caused both by other people's mistakes, and also the inherent challenges of migrating from the error prone POSIX text model to something more reasonable. I *don't* think its reasonable to expect us to do this without support from the corporate users that caused the problem in the first place (by continuing to deploy older versions of Python without investing adequately in their upkeep), so I'd encourage everyone employed by a commercial user of Python to remind their management chains of the risks of failing to invest development time in any upstream dependencies that they expect to keep pace with the dynamic nature of the internet. I hope indeed you are successful in activating resources. However, putting them on this backporting project seems like a waste. They should rather go into porting stuff to 3.x where people need it. Red Hat will be supporting Python 2.6 until 2020'ish, Python 2.7 until at least 2024 (since RHEL7 isn't actually out yet). Python 2 is part of one of our core products (RHEL), and a dependency of another one (OpenStack). While the wheels are in motion upstream to migrate both to Python 3, that's a long term project in both cases, and really hard to make a business case for. While we do believe the open source way delivers better software in the long run, we still have to make our case when we want the company to spend time and resources on something. As responsible maintainers, we should just advise our users that Python 2.7 is a dead horse, and that they should stop riding it. More professionally, we should set an official end-of-life date for 2.7 (alas, we should have done that two years ago). This completely ignores the practical realities of commercial software development, and the commitments that vendors have made to their customers. I agree it would be completely unreasonable to ask volunteers to contribute their own time to making this proposal work. However, it's likely to be much easier for those of us with influence in the commercial world to shake some resources loose if we have a pre-arranged agreement on the specific help we need to better support their interests. I hope that the language summit can agree to stopping bug fix releases for 2.7 in 2014. I'd rather come up with a plan to seek dedicated resources from key corporate users to help with the boring parts of long term maintenance that are really hard to get volunteers excited about, but make a great avenue for corporate sponsored contributions without infringing on the upstream community's autonomy. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
I agree wholeheartedly with the sentiment behind this PEP, but I have concerns about the implementation. If we introduce new APIs into the ssl module then we will see packages and applications that depend on Python 2.7.7+, just like with the introduction of bool in 2.2.1. This will be a mess unless adoption of new Python minor versions is swift and consistent. The PEP says that new features should be testable with feature detection, but how far does that go? The Python 3 NPN interface is a method of SSLContext, which didn't exist in 2.7. Do we backport the entire SSLContext interface in a minor release, or introduce a new NPN interface that is not based on SSLContext just for older Pythons? In my experience most deployments of Python apps have better systems in place to deal with packages installed from PyPI than new versions of Python itself. Therefore it might be better to follow the lead of the unittest2 module and introduce a new ssl2 module that tracks the most recent version of Python's ssl module. My counter-proposal is: * Introduce a new module 'ssl2' which is essentially a backport of the current release's ssl module for all old versions of Python that anyone cares to support. * Third-party applications and libraries targeting anything but the latest version of Python are advised to use ssl2 instead of ssl (this might also be a good opportunity to bless certifi or an equivalent). * Minor releases of Python are allowed to make any security-related change that does not change any public interfaces (e.g. linking against a new version of openssl, or disabling SSLv2) * New interfaces in security-related modules are allowed only if those new interfaces are used by other stdlib modules (since stdlib modules are the only place where ssl2 is off-limits). For example, a backport of match_hostname could go in if httplib or urllib used it. NPN/ALPN would not be allowed under this rule since there is no stdlib code that uses them; third-party SPDY or HTTP/2 code would have to use ssl2 to get this interface. In this world, libraries like requests or tornado that want to use advanced ssl features can simply declare a dependency on ssl2, instead of saying we require Python 2.7.7+, 3.2.6+, etc. In spite of my reservations about this PEP, I'd still love to see SSL improvements in 2.7 in any form. I'm currently planning a port for Tornado's ssl usage from the stdlib ssl module to PyOpenSSL (in order to get SNI and NPN/ALPN features), and I'd really rather not have to convert to a completely incompatible interface. -Ben On Sat, Mar 22, 2014 at 5:11 PM, Nick Coghlan ncogh...@gmail.com wrote: Folks, I have just posted a proposal to change the way we treat enhancements that relate to Python's support for network security enhancements. I now see these as categorically different from most other enhancements, as they have implications for the evolution of internet security as a whole, rather than being limited to affecting the security of individual applications written in Python. It is similar to the previous IDLE policy exception PEP, where we decided that cross version consistency of IDLE superseded the general policy against backporting enhancements to maintenance branches. This new PEP proposes a similar exception for network security related changes, such that the *capabilities* of all still maintained versions should be kept equivalent, with only the *default settings* varying across up to date feature releases. The policy is also designed to make it easier for us to ratchet up the default security settings in future feature releases. However, it's more complex than the IDLE PEP (since it's more invasive), and there are still a few open questions listed. This topic involves a complex balance between encouraging and supporting good security practices and limiting the risk of failures for users upgrading to new maintenance releases, so I'd ask that folks take time to read and consider the implications of the full PEP in the broader context of today's internet before posting comments on specific details, or indicating a preference one way or the other in terms of the overall proposal. Full PEP included inline below, and available in more readable form at http://www.python.org/dev/peps/pep-0466/ Regards, Nick. PEP: 466 Title: Network Security Enhancement Exception for All Branches Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan ncogh...@gmail.com, Status: Draft Type: Informational Content-Type: text/x-rst Created: 23-Mar-2014 Post-History: 23-Mar-2014 Abstract Most CPython tracker issues are classified as errors in behaviour or proposed enhancements. Most patches to fix behavioural errors are applied to all active maintenance branches. Enhancement patches are restricted to the default branch that becomes the next Python version. This PEP relaxes the latter restriction allowing enhancements to be applied to maintenance
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sat, Mar 22, 2014 at 11:26 PM, Martin v. Löwis mar...@v.loewis.dewrote: Am 22.03.14 23:21, schrieb Donald Stufft: Just use Python 3.4 ignores the reality of production software. I wish it were that simple because I love 3.4 But I think so does the PEP (i.e. ignore the reality of production software). The risk of breaking existing installations (which might not be affected by the purported security threat being fixed) is just too high. Users that feel threatened by the lack of SSL features in Python should have long started planning on how to get to Python 3. It's not that they haven't been warned. Or not rely on the standard library for their security. Much as I realize it is necessary for rudimentary SSL support (for example) to exist in the standard library, any security-aware user is much better served by third-party packages they can update more easily than Python. I really wish we had no SSL *server* support in the standard library at all (but then I wish we didn't have so many middling-quality servers in the stdlib to start with :) Even when we're not considering virtualenvs, which are a great tool for preventing upgrades from breaking things, separately-installed packages (be they third-party or official python.org packages) are much easier to upgrade than whole Python installations. And a third-party package like PyOpenSSL has much more freedom in dealing with backward-incompatible changes (because users updating PyOpenSSL are much more likely to realize something regarding SSL could break, even if they aren't more likely to read the release documentation.) I think I'm +0 on the PEP: I understand Martin's concerns, and share them (heck, I'm preparing for an update from Python 2.7.3 to 2.7.6 at work and I'm already having nightmares) but I also understand the problem the PEP tries to solve. I wish we would separate out e.g. the SSL-related modules so they aren't part of the 'core' Python but not quite third-party packages either, but that's not something that's easily done. I do believe that besides accepting this PEP we should provide good guidance on how to write portable, secure software using the tools provided, say in a new chapter in the library docs -- and it shouldn't be afraid to suggest third-party modules if they are a much better solution :) -- Thomas Wouters tho...@python.org Hi! I'm an email virus! Think twice before sending your email to help me spread! ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sat, Mar 22, 2014, at 15:40, Martin v. Löwis wrote: Am 22.03.14 23:33, schrieb Nick Coghlan: Hard to maintain legacy software is a fact of life, and way too much of it is exposed to the public internet. This PEP is about doing what we can to mitigate the damage caused both by other people's mistakes, and also the inherent challenges of migrating from the error prone POSIX text model to something more reasonable. I *don't* think its reasonable to expect us to do this without support from the corporate users that caused the problem in the first place (by continuing to deploy older versions of Python without investing adequately in their upkeep), so I'd encourage everyone employed by a commercial user of Python to remind their management chains of the risks of failing to invest development time in any upstream dependencies that they expect to keep pace with the dynamic nature of the internet. I hope indeed you are successful in activating resources. However, putting them on this backporting project seems like a waste. They should rather go into porting stuff to 3.x where people need it. As responsible maintainers, we should just advise our users that Python 2.7 is a dead horse, and that they should stop riding it. More professionally, we should set an official end-of-life date for 2.7 (alas, we should have done that two years ago). I hope that the language summit can agree to stopping bug fix releases for 2.7 in 2014. As (I believe) previously discussed and documented in PEP 373, this date currently will be May 2015. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
As someone who is deeply biased towards improving the packaging tool chain and getting people to use it I think that most people will simply use the Stdlib even if a more secure alternative exists. Infact one does exist and I still see almost everyone using the stdlib ssl instead of pyopenssl. At best they have an optional dependency on it which many people who aren't security conscious won't even realize why they should install it. On Mar 22, 2014, at 6:53 PM, Ben Darnell b...@bendarnell.com wrote: I agree wholeheartedly with the sentiment behind this PEP, but I have concerns about the implementation. If we introduce new APIs into the ssl module then we will see packages and applications that depend on Python 2.7.7+, just like with the introduction of bool in 2.2.1. This will be a mess unless adoption of new Python minor versions is swift and consistent. The PEP says that new features should be testable with feature detection, but how far does that go? The Python 3 NPN interface is a method of SSLContext, which didn't exist in 2.7. Do we backport the entire SSLContext interface in a minor release, or introduce a new NPN interface that is not based on SSLContext just for older Pythons? In my experience most deployments of Python apps have better systems in place to deal with packages installed from PyPI than new versions of Python itself. Therefore it might be better to follow the lead of the unittest2 module and introduce a new ssl2 module that tracks the most recent version of Python's ssl module. My counter-proposal is: * Introduce a new module 'ssl2' which is essentially a backport of the current release's ssl module for all old versions of Python that anyone cares to support. * Third-party applications and libraries targeting anything but the latest version of Python are advised to use ssl2 instead of ssl (this might also be a good opportunity to bless certifi or an equivalent). * Minor releases of Python are allowed to make any security-related change that does not change any public interfaces (e.g. linking against a new version of openssl, or disabling SSLv2) * New interfaces in security-related modules are allowed only if those new interfaces are used by other stdlib modules (since stdlib modules are the only place where ssl2 is off-limits). For example, a backport of match_hostname could go in if httplib or urllib used it. NPN/ALPN would not be allowed under this rule since there is no stdlib code that uses them; third-party SPDY or HTTP/2 code would have to use ssl2 to get this interface. In this world, libraries like requests or tornado that want to use advanced ssl features can simply declare a dependency on ssl2, instead of saying we require Python 2.7.7+, 3.2.6+, etc. In spite of my reservations about this PEP, I'd still love to see SSL improvements in 2.7 in any form. I'm currently planning a port for Tornado's ssl usage from the stdlib ssl module to PyOpenSSL (in order to get SNI and NPN/ALPN features), and I'd really rather not have to convert to a completely incompatible interface. -Ben On Sat, Mar 22, 2014 at 5:11 PM, Nick Coghlan ncogh...@gmail.com wrote: Folks, I have just posted a proposal to change the way we treat enhancements that relate to Python's support for network security enhancements. I now see these as categorically different from most other enhancements, as they have implications for the evolution of internet security as a whole, rather than being limited to affecting the security of individual applications written in Python. It is similar to the previous IDLE policy exception PEP, where we decided that cross version consistency of IDLE superseded the general policy against backporting enhancements to maintenance branches. This new PEP proposes a similar exception for network security related changes, such that the *capabilities* of all still maintained versions should be kept equivalent, with only the *default settings* varying across up to date feature releases. The policy is also designed to make it easier for us to ratchet up the default security settings in future feature releases. However, it's more complex than the IDLE PEP (since it's more invasive), and there are still a few open questions listed. This topic involves a complex balance between encouraging and supporting good security practices and limiting the risk of failures for users upgrading to new maintenance releases, so I'd ask that folks take time to read and consider the implications of the full PEP in the broader context of today's internet before posting comments on specific details, or indicating a preference one way or the other in terms of the overall proposal. Full PEP included inline below, and available in more readable form at http://www.python.org/dev/peps/pep-0466/ Regards, Nick. PEP: 466 Title: Network Security Enhancement Exception for All
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 23 March 2014 08:53, Ben Darnell b...@bendarnell.com wrote: I agree wholeheartedly with the sentiment behind this PEP, but I have concerns about the implementation. If we introduce new APIs into the ssl module then we will see packages and applications that depend on Python 2.7.7+, just like with the introduction of bool in 2.2.1. This will be a mess unless adoption of new Python minor versions is swift and consistent. The PEP says that new features should be testable with feature detection, but how far does that go? The Python 3 NPN interface is a method of SSLContext, which didn't exist in 2.7. Do we backport the entire SSLContext interface in a minor release, or introduce a new NPN interface that is not based on SSLContext just for older Pythons? The whole thing. The problem at the moment is that too many applications just silently run without proper security settings on older Python release, as users don't completely want to drop support for that version, but also decide it's too hard to enable the right security settings properly. The idea here is to make it easy for them to enable the settings if they're available, but still fall back to their old behaviour if they're not. The fail or fall back decision will depend on the library and its assessment of security compatibility vs breadth of support for legacy versions. The issue isn't really the ssl module itself - it's the other modules that *depend* on the ssl module (like httplib, urllib, poplib, ftplib, imaplib). You could technically try to monkeypatch or shadow the stdlib ssl module from a third party library, but that's a *really* dangerous approach to security (since it makes it hard to do proper integration testing). In my experience most deployments of Python apps have better systems in place to deal with packages installed from PyPI than new versions of Python itself. Therefore it might be better to follow the lead of the unittest2 module and introduce a new ssl2 module that tracks the most recent version of Python's ssl module. My counter-proposal is: * Introduce a new module 'ssl2' which is essentially a backport of the current release's ssl module for all old versions of Python that anyone cares to support. It's not just ssl2: you needs os2, random2, ftplib2, poplib2, imaplib2, smtplib2. You also run headlong into the problem of a complex external dependency on OpenSSL (which the Linux distro repackagers currently handle for us). This is what the reimplementing substantial portions of the standard library as third party modules to gain access to the newer underlying network security protocols and mechanisms, and then injecting those into the module namespace to override their standard library counterparts. comment is about, but I guess I should expand on that for the benefit of readers not as familiar with the relevant implementation details. * Third-party applications and libraries targeting anything but the latest version of Python are advised to use ssl2 instead of ssl (this might also be a good opportunity to bless certifi or an equivalent). * Minor releases of Python are allowed to make any security-related change that does not change any public interfaces (e.g. linking against a new version of openssl, or disabling SSLv2) New versions of OpenSSL are nowhere near as useful when their constants aren't made available in the Python API. * New interfaces in security-related modules are allowed only if those new interfaces are used by other stdlib modules (since stdlib modules are the only place where ssl2 is off-limits). For example, a backport of match_hostname could go in if httplib or urllib used it. NPN/ALPN would not be allowed under this rule since there is no stdlib code that uses them; third-party SPDY or HTTP/2 code would have to use ssl2 to get this interface. In this world, libraries like requests or tornado that want to use advanced ssl features can simply declare a dependency on ssl2, instead of saying we require Python 2.7.7+, 3.2.6+, etc. There's a reason the developers of those libraries haven't already done that: it's genuinely hard. It's not that I didn't think of this approach, it's that I think it would actually be more difficult and less effective than the approach currently in the PEP. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Am 22.03.14 23:39, schrieb Ned Deily: Regarding the python.org binary installers, I think past practice has been for us to update third-party libraries as necessary in maintenance releases when there is good cause and with the concurrence of the release manager, so I don't see this as a big issue. For the OS X binary installer, the issue for OpenSSL has been that we dynamically link to the system-supplied OpenSSL libraries and that, for various reasons, Apple has deprecated (and frozen at non-current OpenSSL releases) the use of those libraries in favor of their own security frameworks. So, for multiple reasons, including the risk that OpenSSL may be dropped from an upcoming major release of OS X, we need to start supplying our own version with all OS X binary installers. That's the plan regardless of the outcome of this PEP. The Windows case is similar but different. I also update OpenSSL releases between Python bug fix releases, but only to OpenSSL bug fix releases (likewise for all other third party software). So 2.7 uses 0.9.8 (currently 0.9.8y), 3.3 and 3.4 use 1.0.1 (currently 1.0.1e). I still wish it was possible to drop OpenSSL altogether on both OSX and Windows, and use the vendor TLS libraries instead. This would reduce the maintenance burden, and might even improve performance. However, it's getting difficult since the ssl module exposes more and more OpenSSL API as-is (rather than trying to provide an abstraction). For Windows, this approach was unfortunately hopeless as long as we had to support XP. Now that Vista becomes the baseline, I hope I can try to provide much of the ssl module natively using CryptoAPI. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Am 23.03.14 00:02, schrieb Benjamin Peterson: As (I believe) previously discussed and documented in PEP 373, this date currently will be May 2015. Ah! Thanks for reminding me. I think PEP 466 then needs to take a position on that, as well. By the past schedule, this probably means two more bug fix releases for 2.7 (which is fine with me). However, I expect this to be a problem for the PEP, since it is unrealistic to expect that all aspects of the desired features are complete by 2015 (i.e. with no need for further updates in the coming years). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 22 March 2014 23:07, Donald Stufft don...@stufft.io wrote: As someone who is deeply biased towards improving the packaging tool chain and getting people to use it I think that most people will simply use the Stdlib even if a more secure alternative exists. Infact one does exist and I still see almost everyone using the stdlib ssl instead of pyopenssl. At best they have an optional dependency on it which many people who aren't security conscious won't even realize why they should install it. Windows users typically will not be able to use something like pyopenssl. It's a complex binary dependency with no wheel on PyPI. There are no easily locatable wininst installers, even - and those are messy to use in a virtualenv. While the stdlib modules may have issues, depend on pyopenssl is not a practical solution for many people. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 23 March 2014 07:32, Benjamin Peterson benja...@python.org wrote: On Sat, Mar 22, 2014, at 14:11, Nick Coghlan wrote: Folks, I have just posted a proposal to change the way we treat enhancements that relate to Python's support for network security enhancements. I think the PEP should also address security-mode releases. Do the same exceptions apply? Does anyone really want to backport features to Python 3.1? As I see it, RHEL is the primary culprit for people still caring about Python 2.6. With the RHEL7 beta published late last year and hence CentOS 7 and RHEL 7 itself presumably landing well before Python 2.7 enters security fix only mode, I think it's OK to leave this aspect in the hands of the distro vendors and other commercial redistributors. We create the problem, we can bear the cost of dealing with it on behalf of our users. In Red Hat's specific case, we're also finally starting to offer users better solutions that make it easier to switch to a newer language runtime without having to upgrade the entire underlying OS or run in an unsupported configuration. In the longer term, this will hopefully mean that the expectation for upstream projects to maintain compatibility with the system Python in RHEL will become limited to just those projects that we actually include as part of the distro. However, I agree the PEP needs to have a dedicated section discussing security fix only releases. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 23 March 2014 09:07, Donald Stufft don...@stufft.io wrote: As someone who is deeply biased towards improving the packaging tool chain and getting people to use it I think that most people will simply use the Stdlib even if a more secure alternative exists. Infact one does exist and I still see almost everyone using the stdlib ssl instead of pyopenssl. At best they have an optional dependency on it which many people who aren't security conscious won't even realize why they should install it. I should probably mention explicitly in the PEP that security related packages almost always involve somewhat tricky binary dependencies, and while we're working on it, our packaging ecosystem will still sometimes deliver a poor user experience on that front. There's also an inherent divide between using an old version of Python and willing to use new dependencies from pip. They're not disjoint sets (plenty of folks use the CentOS system Python + pip), but I expect there's a large enough population of conservative corporate users on older versions that won't use a pip based solution even if its available, thus not really solving the problem. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sat, 22 Mar 2014 19:07:51 -0400 Donald Stufft don...@stufft.io wrote: As someone who is deeply biased towards improving the packaging tool chain and getting people to use it I think that most people will simply use the Stdlib even if a more secure alternative exists. Infact one does exist and I still see almost everyone using the stdlib ssl instead of pyopenssl. pyopenssl exposes a different API from ssl, though. It's not just a matter of flipping an import statement. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sun, 23 Mar 2014 09:08:29 +1000 Nick Coghlan ncogh...@gmail.com wrote: On 23 March 2014 08:53, Ben Darnell b...@bendarnell.com wrote: I agree wholeheartedly with the sentiment behind this PEP, but I have concerns about the implementation. If we introduce new APIs into the ssl module then we will see packages and applications that depend on Python 2.7.7+, just like with the introduction of bool in 2.2.1. This will be a mess unless adoption of new Python minor versions is swift and consistent. The PEP says that new features should be testable with feature detection, but how far does that go? The Python 3 NPN interface is a method of SSLContext, which didn't exist in 2.7. Do we backport the entire SSLContext interface in a minor release, or introduce a new NPN interface that is not based on SSLContext just for older Pythons? The whole thing. Have you first tried to do it and appreciate the amount of disruption it would cause? It's not just ssl2: you needs os2, random2, ftplib2, poplib2, imaplib2, smtplib2. You also run headlong into the problem of a complex external dependency on OpenSSL (which the Linux distro repackagers currently handle for us). So are you proposing to backport changes to all these modules too? This sounds crazy. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sat, Mar 22, 2014, at 16:34, Antoine Pitrou wrote: On Sun, 23 Mar 2014 09:08:29 +1000 Nick Coghlan ncogh...@gmail.com wrote: On 23 March 2014 08:53, Ben Darnell b...@bendarnell.com wrote: I agree wholeheartedly with the sentiment behind this PEP, but I have concerns about the implementation. If we introduce new APIs into the ssl module then we will see packages and applications that depend on Python 2.7.7+, just like with the introduction of bool in 2.2.1. This will be a mess unless adoption of new Python minor versions is swift and consistent. The PEP says that new features should be testable with feature detection, but how far does that go? The Python 3 NPN interface is a method of SSLContext, which didn't exist in 2.7. Do we backport the entire SSLContext interface in a minor release, or introduce a new NPN interface that is not based on SSLContext just for older Pythons? The whole thing. Have you first tried to do it and appreciate the amount of disruption it would cause? It's not just ssl2: you needs os2, random2, ftplib2, poplib2, imaplib2, smtplib2. You also run headlong into the problem of a complex external dependency on OpenSSL (which the Linux distro repackagers currently handle for us). So are you proposing to backport changes to all these modules too? This sounds crazy. If one reads the PEP, one will notice it indeed proposes a conditional exception for networking related modules that depend on one or more of the network security related modules [ssl, hashlib,...]. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sat, Mar 22, 2014 at 7:08 PM, Nick Coghlan ncogh...@gmail.com wrote: The issue isn't really the ssl module itself - it's the other modules that *depend* on the ssl module (like httplib, urllib, poplib, ftplib, imaplib). You could technically try to monkeypatch or shadow the stdlib ssl module from a third party library, but that's a *really* dangerous approach to security (since it makes it hard to do proper integration testing). I suppose the difference is that the applications I've worked with have *already* abandoned stdlib modules like httplib or urllib in favor of libraries like requests and tornado, so I see this as mainly a question of whether the stdlib provides a suitable foundation for those packages. The argument about the difficulty of integration testing cuts both ways - I can use tox to test Tornado with several versions of a third-party package, but cannot (as long as https://launchpad.net/~fkrull/+archive/deadsnakesonly provides one package per x.y release) easily test with both Python 2.7.6 and 2.7.7. In my experience most deployments of Python apps have better systems in place to deal with packages installed from PyPI than new versions of Python itself. Therefore it might be better to follow the lead of the unittest2 module and introduce a new ssl2 module that tracks the most recent version of Python's ssl module. My counter-proposal is: * Introduce a new module 'ssl2' which is essentially a backport of the current release's ssl module for all old versions of Python that anyone cares to support. It's not just ssl2: you needs os2, random2, ftplib2, poplib2, imaplib2, smtplib2. You also run headlong into the problem of a complex external dependency on OpenSSL (which the Linux distro repackagers currently handle for us). Just os2 and random2; as I said a few bullet points down the stdlib libraries that use ssl would be sufficient justification for additions to the ssl module's interface. This is what the reimplementing substantial portions of the standard library as third party modules to gain access to the newer underlying network security protocols and mechanisms, and then injecting those into the module namespace to override their standard library counterparts. comment is about, but I guess I should expand on that for the benefit of readers not as familiar with the relevant implementation details. In my proposal there is no monkey-patching; the newer ssl interfaces would come solely from the ssl2 module. * Third-party applications and libraries targeting anything but the latest version of Python are advised to use ssl2 instead of ssl (this might also be a good opportunity to bless certifi or an equivalent). * Minor releases of Python are allowed to make any security-related change that does not change any public interfaces (e.g. linking against a new version of openssl, or disabling SSLv2) New versions of OpenSSL are nowhere near as useful when their constants aren't made available in the Python API. * New interfaces in security-related modules are allowed only if those new interfaces are used by other stdlib modules (since stdlib modules are the only place where ssl2 is off-limits). For example, a backport of match_hostname could go in if httplib or urllib used it. NPN/ALPN would not be allowed under this rule since there is no stdlib code that uses them; third-party SPDY or HTTP/2 code would have to use ssl2 to get this interface. In this world, libraries like requests or tornado that want to use advanced ssl features can simply declare a dependency on ssl2, instead of saying we require Python 2.7.7+, 3.2.6+, etc. There's a reason the developers of those libraries haven't already done that: it's genuinely hard. It's not that I didn't think of this approach, it's that I think it would actually be more difficult and less effective than the approach currently in the PEP. One of the reasons it's genuinely hard is that no one has provided an interface compatible with the stdlib ssl module. An ssl2 module that backported Python 3.4's ssl module would be a *much* more attractive porting target than e.g. PyOpenSSL. -Ben Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
In the case of requests they already have an optional dependency on pyopenssl. It's just many people either don't know they should use it, are unable to use it, or unwilling to use the python packaging tool chain because of its current flaws. On Mar 22, 2014, at 7:42 PM, Ben Darnell b...@bendarnell.com wrote: On Sat, Mar 22, 2014 at 7:08 PM, Nick Coghlan ncogh...@gmail.com wrote: The issue isn't really the ssl module itself - it's the other modules that *depend* on the ssl module (like httplib, urllib, poplib, ftplib, imaplib). You could technically try to monkeypatch or shadow the stdlib ssl module from a third party library, but that's a *really* dangerous approach to security (since it makes it hard to do proper integration testing). I suppose the difference is that the applications I've worked with have *already* abandoned stdlib modules like httplib or urllib in favor of libraries like requests and tornado, so I see this as mainly a question of whether the stdlib provides a suitable foundation for those packages. The argument about the difficulty of integration testing cuts both ways - I can use tox to test Tornado with several versions of a third-party package, but cannot (as long as https://launchpad.net/~fkrull/+archive/deadsnakes only provides one package per x.y release) easily test with both Python 2.7.6 and 2.7.7. In my experience most deployments of Python apps have better systems in place to deal with packages installed from PyPI than new versions of Python itself. Therefore it might be better to follow the lead of the unittest2 module and introduce a new ssl2 module that tracks the most recent version of Python's ssl module. My counter-proposal is: * Introduce a new module 'ssl2' which is essentially a backport of the current release's ssl module for all old versions of Python that anyone cares to support. It's not just ssl2: you needs os2, random2, ftplib2, poplib2, imaplib2, smtplib2. You also run headlong into the problem of a complex external dependency on OpenSSL (which the Linux distro repackagers currently handle for us). Just os2 and random2; as I said a few bullet points down the stdlib libraries that use ssl would be sufficient justification for additions to the ssl module's interface. This is what the reimplementing substantial portions of the standard library as third party modules to gain access to the newer underlying network security protocols and mechanisms, and then injecting those into the module namespace to override their standard library counterparts. comment is about, but I guess I should expand on that for the benefit of readers not as familiar with the relevant implementation details. In my proposal there is no monkey-patching; the newer ssl interfaces would come solely from the ssl2 module. * Third-party applications and libraries targeting anything but the latest version of Python are advised to use ssl2 instead of ssl (this might also be a good opportunity to bless certifi or an equivalent). * Minor releases of Python are allowed to make any security-related change that does not change any public interfaces (e.g. linking against a new version of openssl, or disabling SSLv2) New versions of OpenSSL are nowhere near as useful when their constants aren't made available in the Python API. * New interfaces in security-related modules are allowed only if those new interfaces are used by other stdlib modules (since stdlib modules are the only place where ssl2 is off-limits). For example, a backport of match_hostname could go in if httplib or urllib used it. NPN/ALPN would not be allowed under this rule since there is no stdlib code that uses them; third-party SPDY or HTTP/2 code would have to use ssl2 to get this interface. In this world, libraries like requests or tornado that want to use advanced ssl features can simply declare a dependency on ssl2, instead of saying we require Python 2.7.7+, 3.2.6+, etc. There's a reason the developers of those libraries haven't already done that: it's genuinely hard. It's not that I didn't think of this approach, it's that I think it would actually be more difficult and less effective than the approach currently in the PEP. One of the reasons it's genuinely hard is that no one has provided an interface compatible with the stdlib ssl module. An ssl2 module that backported Python 3.4's ssl module would be a *much* more attractive porting target than e.g. PyOpenSSL. -Ben Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io ___ Python-Dev mailing list
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 22 March 2014 23:49, Donald Stufft don...@stufft.io wrote: In the case of requests they already have an optional dependency on pyopenssl. It's just many people either don't know they should use it, are unable to use it, or unwilling to use the python packaging tool chain because of its current flaws. Do they use the new features in the Python 3.x ssl module when it's available to give the same level of security as having pyopenssl would, or do they use a lowest common denominator (i.e., 2.x compatible) level of security when using the stdlib? If the latter, that would be very, very sad. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Also important to remember that pip itself uses the OpenSSL binding in the ssl module so there is a chicken and egg problem. On Mar 22, 2014, at 7:49 PM, Donald Stufft don...@stufft.io wrote: In the case of requests they already have an optional dependency on pyopenssl. It's just many people either don't know they should use it, are unable to use it, or unwilling to use the python packaging tool chain because of its current flaws. On Mar 22, 2014, at 7:42 PM, Ben Darnell b...@bendarnell.com wrote: On Sat, Mar 22, 2014 at 7:08 PM, Nick Coghlan ncogh...@gmail.com wrote: The issue isn't really the ssl module itself - it's the other modules that *depend* on the ssl module (like httplib, urllib, poplib, ftplib, imaplib). You could technically try to monkeypatch or shadow the stdlib ssl module from a third party library, but that's a *really* dangerous approach to security (since it makes it hard to do proper integration testing). I suppose the difference is that the applications I've worked with have *already* abandoned stdlib modules like httplib or urllib in favor of libraries like requests and tornado, so I see this as mainly a question of whether the stdlib provides a suitable foundation for those packages. The argument about the difficulty of integration testing cuts both ways - I can use tox to test Tornado with several versions of a third-party package, but cannot (as long as https://launchpad.net/~fkrull/+archive/deadsnakes only provides one package per x.y release) easily test with both Python 2.7.6 and 2.7.7. In my experience most deployments of Python apps have better systems in place to deal with packages installed from PyPI than new versions of Python itself. Therefore it might be better to follow the lead of the unittest2 module and introduce a new ssl2 module that tracks the most recent version of Python's ssl module. My counter-proposal is: * Introduce a new module 'ssl2' which is essentially a backport of the current release's ssl module for all old versions of Python that anyone cares to support. It's not just ssl2: you needs os2, random2, ftplib2, poplib2, imaplib2, smtplib2. You also run headlong into the problem of a complex external dependency on OpenSSL (which the Linux distro repackagers currently handle for us). Just os2 and random2; as I said a few bullet points down the stdlib libraries that use ssl would be sufficient justification for additions to the ssl module's interface. This is what the reimplementing substantial portions of the standard library as third party modules to gain access to the newer underlying network security protocols and mechanisms, and then injecting those into the module namespace to override their standard library counterparts. comment is about, but I guess I should expand on that for the benefit of readers not as familiar with the relevant implementation details. In my proposal there is no monkey-patching; the newer ssl interfaces would come solely from the ssl2 module. * Third-party applications and libraries targeting anything but the latest version of Python are advised to use ssl2 instead of ssl (this might also be a good opportunity to bless certifi or an equivalent). * Minor releases of Python are allowed to make any security-related change that does not change any public interfaces (e.g. linking against a new version of openssl, or disabling SSLv2) New versions of OpenSSL are nowhere near as useful when their constants aren't made available in the Python API. * New interfaces in security-related modules are allowed only if those new interfaces are used by other stdlib modules (since stdlib modules are the only place where ssl2 is off-limits). For example, a backport of match_hostname could go in if httplib or urllib used it. NPN/ALPN would not be allowed under this rule since there is no stdlib code that uses them; third-party SPDY or HTTP/2 code would have to use ssl2 to get this interface. In this world, libraries like requests or tornado that want to use advanced ssl features can simply declare a dependency on ssl2, instead of saying we require Python 2.7.7+, 3.2.6+, etc. There's a reason the developers of those libraries haven't already done that: it's genuinely hard. It's not that I didn't think of this approach, it's that I think it would actually be more difficult and less effective than the approach currently in the PEP. One of the reasons it's genuinely hard is that no one has provided an interface compatible with the stdlib ssl module. An ssl2 module that backported Python 3.4's ssl module would be a *much* more attractive porting target than e.g. PyOpenSSL. -Ben Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
They detect for ssl to have the SSLContext and use it if it's available. On Mar 22, 2014, at 7:54 PM, Paul Moore p.f.mo...@gmail.com wrote: On 22 March 2014 23:49, Donald Stufft don...@stufft.io wrote: In the case of requests they already have an optional dependency on pyopenssl. It's just many people either don't know they should use it, are unable to use it, or unwilling to use the python packaging tool chain because of its current flaws. Do they use the new features in the Python 3.x ssl module when it's available to give the same level of security as having pyopenssl would, or do they use a lowest common denominator (i.e., 2.x compatible) level of security when using the stdlib? If the latter, that would be very, very sad. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sun, Mar 23, 2014 at 10:33 AM, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 22 Mar 2014 19:07:51 -0400 Donald Stufft don...@stufft.io wrote: As someone who is deeply biased towards improving the packaging tool chain and getting people to use it I think that most people will simply use the Stdlib even if a more secure alternative exists. Infact one does exist and I still see almost everyone using the stdlib ssl instead of pyopenssl. pyopenssl exposes a different API from ssl, though. It's not just a matter of flipping an import statement. How hard would it be to create a wrapper module named ssl.py, which imports pyopenssl (thus taking advantage of any updates it gets), and could be placed earlier in the search path than the stdlib one? I'm +1 on the overall concept, but not sure about 2.6 and older. 2.7 is already a bit of an odd duck; it's being supported for longer than its official Windows compiler, it's going to be supported by Red Hat until eternity and beyond, and it wouldn't in the least surprise me if 2.7.10 introduces a bit of stuff, backported, in ways that are fundamentally security fixes. Is it really that bad to have something depend on 2.7.10 rather than 2.7.0? If a program will bomb without some bugfix, then it's not unreasonable to say that it depends on a version of Python that includes that fix. As long as any program written for 2.7.0 will run on 2.7.x unchanged (the normal expectations with semantic versioning), I don't see a problem with some of the bug fixes actually adding attributes; imagine if some feature attribute were simply misspelled in the previous versions, which could surely be fixed in a point release; it's not going to break anything. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sun, 23 Mar 2014 07:11:07 +1000 Nick Coghlan ncogh...@gmail.com wrote: It is similar to the previous IDLE policy exception PEP, where we decided that cross version consistency of IDLE superseded the general policy against backporting enhancements to maintenance branches. But the consequences are different: the stdlib promises API stability between bugfix releases, while IDLE is a standalone GUI application. This topic involves a complex balance between encouraging and supporting good security practices and limiting the risk of failures for users upgrading to new maintenance releases, so I'd ask that folks take time to read and consider the implications of the full PEP in the broader context of today's internet before posting comments on specific details, or indicating a preference one way or the other in terms of the overall proposal. The fact that it involves a complex balance implies IMO that a blanket exemption is the wrong mechanism, and instead the exemption should probably be granted on a case-by-case basis. This PEP does *not* grant any general exemptions to the usual backwards compatibility policy for maintenance releases. Instead, it is designed to make it easier to provide more secure by default behaviour in future feature releases, while still limiting the risk of breaking currently working software when upgrading to a new maintenance release. But enforcing secure by default can by construction break backwards compatibility, which is the very reason we are so conservative with such changes. Thus the focus on network security protocols and related cryptographic infrastructure - This is a bit limited. There are remotely-exploitable security issues which are still open on the bug tracker; they are not cryptography-related, but that shouldn't make a difference. (for example the dreaded XML issues have never been properly fixed, AFAICT) My position on the ongoing transition from Python 2 to Python 3 has long been that Python 2 remains a supported platform for the core development team, and that commercial support will remain available well after upstream maintenance ends. If this is about the difficulty of migrating to Python 3, then this PEP should focus specifically on that problem, and restrict the policy to Python 2.7. Upgrading from 3.X to 3.X+1 is easy, and so is backporting patches from 3.X to 3.X-1, conversely. * Are there any other security relevant modules that should be covered by either a blanket or conditional exemption? Every module can be remotely security relevant, unfortunately. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sat, 22 Mar 2014 23:54:37 +0100 Thomas Wouters tho...@python.org wrote: Or not rely on the standard library for their security. Much as I realize it is necessary for rudimentary SSL support (for example) to exist in the standard library, Unfortunately, rudimentary SSL support is worse than nothing. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sun, 23 Mar 2014 10:59:22 +1100 Chris Angelico ros...@gmail.com wrote: Is it really that bad to have something depend on 2.7.10 rather than 2.7.0? It makes our whole release and versioning scheme very confusing (hopefully people will not have to read Nick's PEP to understand it :-)). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 23.03.2014 01:01, Antoine Pitrou wrote: This is a bit limited. There are remotely-exploitable security issues which are still open on the bug tracker; they are not cryptography-related, but that shouldn't make a difference. (for example the dreaded XML issues have never been properly fixed, AFAICT) True, you may blame me for the situation. Only a handful of people were interested in the XML issues. I ran out of steam and moved to more sapid topics, too. Christian ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 23 Mar 2014 10:18, Christian Heimes christ...@python.org wrote: On 23.03.2014 01:01, Antoine Pitrou wrote: This is a bit limited. There are remotely-exploitable security issues which are still open on the bug tracker; they are not cryptography-related, but that shouldn't make a difference. (for example the dreaded XML issues have never been properly fixed, AFAICT) True, you may blame me for the situation. Only a handful of people were interested in the XML issues. I ran out of steam and moved to more sapid topics, too. The key difference there is we're not reliant on defusedxml to download defusedxml :) Anyway, folks have given me plenty of good feedback to chew over. I'm going to go run some errands and then try to get a new revision posted this afternoon. Cheers, Nick. Christian ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sun, 23 Mar 2014 01:01:38 +0100, Antoine Pitrou solip...@pitrou.net wrote: On Sun, 23 Mar 2014 07:11:07 +1000 Nick Coghlan ncogh...@gmail.com wrote: This PEP does *not* grant any general exemptions to the usual backwards compatibility policy for maintenance releases. Instead, it is designed to make it easier to provide more secure by default behaviour in future feature releases, while still limiting the risk of breaking currently working software when upgrading to a new maintenance release. But enforcing secure by default can by construction break backwards compatibility, which is the very reason we are so conservative with such changes. Yeah, I couldn't figure out what this paragraph meant. What does backporting features in maintenance releases have to do with making feature releases secure by default? If we are not relaxing the backward compatibility rules we can't do secure by default changes in a maintenance release...and if we don't do that, thinking that very much legacy software is going to get upgraded to use the new more secure capabilities is, I suspect (but cannot prove, obviously) mostly wishful thinking. An exception might be be commercial packages, but even there I wouldn't bet very many bucks. It seems to me that shops who are being security concious enough to be candidates for modifying currently deployed packages to be more secure are also likely to have already addressed the security issues in some other way. Now, requests and other libraries could use these new features, and since they have faster release cycles, they could presumably make the increased security be the default sooner. So that's what I see as the argument in favor...but it is a much narrower impact area than most python software that uses ssl, and certainly doesn't cover poplib, smtplib, etc. But perhaps I am being overly cynical. Regardless, like Martin, I prefer to focus on Python3. --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Am 23.03.14 01:15, schrieb Christian Heimes: On 23.03.2014 01:01, Antoine Pitrou wrote: This is a bit limited. There are remotely-exploitable security issues which are still open on the bug tracker; they are not cryptography-related, but that shouldn't make a difference. (for example the dreaded XML issues have never been properly fixed, AFAICT) True, you may blame me for the situation. Only a handful of people were interested in the XML issues. I ran out of steam and moved to more sapid topics, too I don't think anybody wanted to assign blame (although I can sympathize with your urge to accept the blame). The fact is that this is a volunteer project: we do what we can and have fun doing. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 23 March 2014 10:08, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 22 Mar 2014 23:54:37 +0100 Thomas Wouters tho...@python.org wrote: Or not rely on the standard library for their security. Much as I realize it is necessary for rudimentary SSL support (for example) to exist in the standard library, Unfortunately, rudimentary SSL support is worse than nothing. I'm going to try to find a way to steal that line and get it into the PEP. I'm not sure yet if my proposal is the *right* answer, but this observation gets right to the heart of the problem that actually needs to be solved. I'd actually welcome a competing PEP that seriously explored separating out the relevant parts of the Python 3.4 standard library as a PyPI project and attempted to tackle the problem by making that available at least as far back as 2.7 and then just deprecating the leaking batteries in the 2.7 standard library. Just because I think that approach would be even more work and even more disruptive than what I propose in PEP 466, doesn't mean it isn't a better idea overall. What we have essentially found is that where we could basically get away with an 18 month update cycle for improved network security support (extended out to a few years by certain major platform vendors), that approach *isn't* working when it comes to putting a feature release into long term maintenance mode. I don't think the situation isn't critical yet, but it's getting close, and I think we need to deal with it within the 12 months (and preferably sooner than that). My PEP proposes one way of addressing the challenge: allowing a wider variety of changes in maintenance releases to reduce the variance in capability in this area. Moving the affected modules out of the standard library proper and bundling the critical ones along with pip instead is indeed another alternative. However, that approach introduces additional issues of its own - I'll cover some of them in the next PEP update, but it would be good to have someone explicitly trying to make the case that a PyPI backport would be simpler for the overall ecosystem than my suggested approach. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 23 March 2014 10:40, Martin v. Löwis mar...@v.loewis.de wrote: Am 23.03.14 01:15, schrieb Christian Heimes: On 23.03.2014 01:01, Antoine Pitrou wrote: This is a bit limited. There are remotely-exploitable security issues which are still open on the bug tracker; they are not cryptography-related, but that shouldn't make a difference. (for example the dreaded XML issues have never been properly fixed, AFAICT) True, you may blame me for the situation. Only a handful of people were interested in the XML issues. I ran out of steam and moved to more sapid topics, too I don't think anybody wanted to assign blame (although I can sympathize with your urge to accept the blame). The fact is that this is a volunteer project: we do what we can and have fun doing. Agreed completely - what I'm trying to do here is set up a plan that is at least *acceptable* to the upstream community, so we can then seek corporate support for actually putting it into practice (and I'd advise against us accepting *any* proposal to resolve the situation without receiving binding commitments to provide ongoing maintenance support - while I think this proposal is important, I'm under no illusions that actually implementing it will be fun, and it's not appropriate to ask people to do that in their own time). However, we have a *lot* of downstream users and redistributors that have been taking CPython core development for granted, and by so doing, they have allowed a situation to develop that has some rather negative implications for the overall security of networked communications in the Python ecosystem. Since some of those same corporate redistributors are a key enabler allowing users to stay on those old releases that are no longer supported upstream, and others are likely to be being conservative in their own Python 3 migrations, I believe they share a lot of the responsibility for helping to resolve it, either by facilitating the migration to Python 3, helping to improve the networking security situation in Python 2, or, preferably, both. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sun, 23 Mar 2014 01:40:32 +0100 Martin v. Löwis mar...@v.loewis.de wrote: Am 23.03.14 01:15, schrieb Christian Heimes: On 23.03.2014 01:01, Antoine Pitrou wrote: This is a bit limited. There are remotely-exploitable security issues which are still open on the bug tracker; they are not cryptography-related, but that shouldn't make a difference. (for example the dreaded XML issues have never been properly fixed, AFAICT) True, you may blame me for the situation. Only a handful of people were interested in the XML issues. I ran out of steam and moved to more sapid topics, too I don't think anybody wanted to assign blame (although I can sympathize with your urge to accept the blame). The fact is that this is a volunteer project: we do what we can and have fun doing. Indeed this wasn't meant to blame anyone, simply to point out that improving security is often hampered by lack of manpower or motivation. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Mar 22, 2014, at 8:55 PM, Nick Coghlan ncogh...@gmail.com wrote: Moving the affected modules out of the standard library proper and bundling the critical ones along with pip instead is indeed another alternative. However, that approach introduces additional issues of its own - I'll cover some of them in the next PEP update, but it would be good to have someone explicitly trying to make the case that a PyPI backport would be simpler for the overall ecosystem than my suggested approach. FWIW pip as of right now has a policy of no C dependencies outside of the stdlib. CPython isn’t our only target and C dependencies don’t work very well on PyPy (if at all) and it makes the situation much more difficult on platforms where there are no compiler toolchains (Windows). - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
I'm a bit under the weather and I'm not sure what to think of this yet. With that provision, and trying to be brief: I agree that there are security concerns about Python 2.7 that can't be addressed by recommending Python 3.4 instead. I also agree that the ban on new features in old releases can be stifling. But I also worry about breaking working code. I am a bit worried that writing a PEP (at least this specific PEP) would become less about arguing for a specific technical (or even political) idea, and more about putting weasel words in an official document so that it can be used to justify future actions without due process. I wonder whether there is a specific narrower proposal (e.g. add SSLContext to Python 2.7.7?) that would make the discussion more concrete. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sat Mar 22 2014 at 8:55:36 PM, Nick Coghlan ncogh...@gmail.com wrote: On 23 March 2014 10:08, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 22 Mar 2014 23:54:37 +0100 Thomas Wouters tho...@python.org wrote: Or not rely on the standard library for their security. Much as I realize it is necessary for rudimentary SSL support (for example) to exist in the standard library, Unfortunately, rudimentary SSL support is worse than nothing. I'm going to try to find a way to steal that line and get it into the PEP. I'm not sure yet if my proposal is the *right* answer, but this observation gets right to the heart of the problem that actually needs to be solved. I'd actually welcome a competing PEP that seriously explored separating out the relevant parts of the Python 3.4 standard library as a PyPI project and attempted to tackle the problem by making that available at least as far back as 2.7 and then just deprecating the leaking batteries in the 2.7 standard library. Now I have been reading this thread on my phone and I only have cursory understanding of what failure ssl has had as of late, so this might be stupid, but what if in Python 3.5 we made it so people passed in an explicit SSL object into the relevant APIs and threw an exception if one wasn't provided when trying to access something requiring SSL? Then the ssl module would start existing on PyPI to make sure it can evolve faster than every 18 months? I know this doesn't help older APIs but it would make it rather explicit going forward that if you want security you need the latest thing and that's found on PyPI so it would at least solve the issue for the future which is better than nothing. We could try to provide matching APIs that monkey-patch older versions of Python if that's so desired to make this backwards-compatible, but as we all know that can be rather brittle. -Brett Just because I think that approach would be even more work and even more disruptive than what I propose in PEP 466, doesn't mean it isn't a better idea overall. What we have essentially found is that where we could basically get away with an 18 month update cycle for improved network security support (extended out to a few years by certain major platform vendors), that approach *isn't* working when it comes to putting a feature release into long term maintenance mode. I don't think the situation isn't critical yet, but it's getting close, and I think we need to deal with it within the 12 months (and preferably sooner than that). My PEP proposes one way of addressing the challenge: allowing a wider variety of changes in maintenance releases to reduce the variance in capability in this area. Moving the affected modules out of the standard library proper and bundling the critical ones along with pip instead is indeed another alternative. However, that approach introduces additional issues of its own - I'll cover some of them in the next PEP update, but it would be good to have someone explicitly trying to make the case that a PyPI backport would be simpler for the overall ecosystem than my suggested approach. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ brett%40python.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Mar 22, 2014, at 9:17 PM, Ben Darnell b...@bendarnell.com wrote: On Sat, Mar 22, 2014 at 8:55 PM, Nick Coghlan ncogh...@gmail.com wrote: What we have essentially found is that where we could basically get away with an 18 month update cycle for improved network security support (extended out to a few years by certain major platform vendors), that approach *isn't* working when it comes to putting a feature release into long term maintenance mode. I don't think the situation isn't critical yet, but it's getting close, and I think we need to deal with it within the 12 months (and preferably sooner than that). This PEP as written applies to both Python 2.x and 3.x, but the two situations are very different. 3.x is on a ~18 month update cycle, so why isn't the status quo acceptable there? Python 2.x has less than 18 months of support left, so could it get by with a single exceptional release instead of a general relaxing of the rules? (if it were up to me, I'd call that release Python 2.8 instead of 2.7.7) If this PEP is mainly about a one-shot update to the security components of Python 2.x, I'd like to see an explicit list of what is in scope for the update. -Ben ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io As one of the instigators who convinced Nick to write this PEP I’m far less concerned about back porting things to earlier 3.x releases than I am about getting things into 2.7 at this point. Going from 3.2 to 3.4 is not terribly difficult if it requires any work at all. Going from 2.7 to 3.4 is often times a significant investment in resources that has to be taken by *every* network using project. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Mar 22, 2014, at 9:33 PM, Brett Cannon bcan...@gmail.com wrote: On Sat Mar 22 2014 at 8:55:36 PM, Nick Coghlan ncogh...@gmail.com wrote: On 23 March 2014 10:08, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 22 Mar 2014 23:54:37 +0100 Thomas Wouters tho...@python.org wrote: Or not rely on the standard library for their security. Much as I realize it is necessary for rudimentary SSL support (for example) to exist in the standard library, Unfortunately, rudimentary SSL support is worse than nothing. I'm going to try to find a way to steal that line and get it into the PEP. I'm not sure yet if my proposal is the *right* answer, but this observation gets right to the heart of the problem that actually needs to be solved. I'd actually welcome a competing PEP that seriously explored separating out the relevant parts of the Python 3.4 standard library as a PyPI project and attempted to tackle the problem by making that available at least as far back as 2.7 and then just deprecating the leaking batteries in the 2.7 standard library. Now I have been reading this thread on my phone and I only have cursory understanding of what failure ssl has had as of late, so this might be stupid, but what if in Python 3.5 we made it so people passed in an explicit SSL object into the relevant APIs and threw an exception if one wasn't provided when trying to access something requiring SSL? Then the ssl module would start existing on PyPI to make sure it can evolve faster than every 18 months? It’s not really a failure so much as what was concerned secure in 2010 isn’t the case in 2014. I don’t want to sounds like I’m trying to say that pip should dictate Python policy or anything but pip more or less depends on the fact that ssl is available in the stdlib at this point in time in order to be able to handle secure downloads. Every 18 months isn’t that big of a deal tbh, the larger concern is Python 2.7 which has had nearly 4 years since it’s release and is likely to exist for far more years past that. I know this doesn't help older APIs but it would make it rather explicit going forward that if you want security you need the latest thing and that's found on PyPI so it would at least solve the issue for the future which is better than nothing. We could try to provide matching APIs that monkey-patch older versions of Python if that's so desired to make this backwards-compatible, but as we all know that can be rather brittle. -Brett Just because I think that approach would be even more work and even more disruptive than what I propose in PEP 466, doesn't mean it isn't a better idea overall. What we have essentially found is that where we could basically get away with an 18 month update cycle for improved network security support (extended out to a few years by certain major platform vendors), that approach *isn't* working when it comes to putting a feature release into long term maintenance mode. I don't think the situation isn't critical yet, but it's getting close, and I think we need to deal with it within the 12 months (and preferably sooner than that). My PEP proposes one way of addressing the challenge: allowing a wider variety of changes in maintenance releases to reduce the variance in capability in this area. Moving the affected modules out of the standard library proper and bundling the critical ones along with pip instead is indeed another alternative. However, that approach introduces additional issues of its own - I'll cover some of them in the next PEP update, but it would be good to have someone explicitly trying to make the case that a PyPI backport would be simpler for the overall ecosystem than my suggested approach. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/brett%40python.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 23 Mar 2014 11:47, Donald Stufft don...@stufft.io wrote: On Mar 22, 2014, at 9:33 PM, Brett Cannon bcan...@gmail.com wrote: On Sat Mar 22 2014 at 8:55:36 PM, Nick Coghlan ncogh...@gmail.com wrote: On 23 March 2014 10:08, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 22 Mar 2014 23:54:37 +0100 Thomas Wouters tho...@python.org wrote: Or not rely on the standard library for their security. Much as I realize it is necessary for rudimentary SSL support (for example) to exist in the standard library, Unfortunately, rudimentary SSL support is worse than nothing. I'm going to try to find a way to steal that line and get it into the PEP. I'm not sure yet if my proposal is the *right* answer, but this observation gets right to the heart of the problem that actually needs to be solved. I'd actually welcome a competing PEP that seriously explored separating out the relevant parts of the Python 3.4 standard library as a PyPI project and attempted to tackle the problem by making that available at least as far back as 2.7 and then just deprecating the leaking batteries in the 2.7 standard library. Now I have been reading this thread on my phone and I only have cursory understanding of what failure ssl has had as of late, so this might be stupid, but what if in Python 3.5 we made it so people passed in an explicit SSL object into the relevant APIs and threw an exception if one wasn't provided when trying to access something requiring SSL? Then the ssl module would start existing on PyPI to make sure it can evolve faster than every 18 months? It's not really a failure so much as what was concerned secure in 2010 isn't the case in 2014. I don't want to sounds like I'm trying to say that pip should dictate Python policy or anything but pip more or less depends on the fact that ssl is available in the stdlib at this point in time in order to be able to handle secure downloads. Every 18 months isn't that big of a deal tbh, the larger concern is Python 2.7 which has had nearly 4 years since it's release and is likely to exist for far more years past that. Yes, I think it's worth narrowing the scope to just Python 2.7, and leaving the process unchanged for 3.x. Cheers, Nick. I know this doesn't help older APIs but it would make it rather explicit going forward that if you want security you need the latest thing and that's found on PyPI so it would at least solve the issue for the future which is better than nothing. We could try to provide matching APIs that monkey-patch older versions of Python if that's so desired to make this backwards-compatible, but as we all know that can be rather brittle. -Brett Just because I think that approach would be even more work and even more disruptive than what I propose in PEP 466, doesn't mean it isn't a better idea overall. What we have essentially found is that where we could basically get away with an 18 month update cycle for improved network security support (extended out to a few years by certain major platform vendors), that approach *isn't* working when it comes to putting a feature release into long term maintenance mode. I don't think the situation isn't critical yet, but it's getting close, and I think we need to deal with it within the 12 months (and preferably sooner than that). My PEP proposes one way of addressing the challenge: allowing a wider variety of changes in maintenance releases to reduce the variance in capability in this area. Moving the affected modules out of the standard library proper and bundling the critical ones along with pip instead is indeed another alternative. However, that approach introduces additional issues of its own - I'll cover some of them in the next PEP update, but it would be good to have someone explicitly trying to make the case that a PyPI backport would be simpler for the overall ecosystem than my suggested approach. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/brett%40python.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 3/22/2014 8:55 PM, Nick Coghlan wrote: Unfortunately, rudimentary SSL support is worse than nothing. I'm going to try to find a way to steal that line and get it into the PEP. I'm not sure yet if my proposal is the *right* answer, but this observation gets right to the heart of the problem that actually needs to be solved. I'd actually welcome a competing PEP that seriously explored separating out the relevant parts of the Python 3.4 standard library as a PyPI project and attempted to tackle the problem by making that available at least as far back as 2.7 and then just deprecating the leaking batteries in the 2.7 standard library. I have a competing proposal that is a variation on yours. First, restrict attention to 2.7. The problem, as you defined it in the PEP, is that a) 2.7 is currently not an adequate base for public internet servers, coupled with b) some companies are determined to continue using 2.x for their servers, and c) condition b) is not going to end in a year or two. Your proposal is to enhance 2.7 by backporting some indefinite number of server security features from 3.x to 2.7. The first problem with this idea is that the result will not be Python 2.7 as currently defined. So lets not break our naming and stability policy and and engender legitimate objections by labeling Server Security Python as Python 2.7.z, where z is a single digit. The second problem is that enhancement will not, I strongly suspect, be a one time event. So there will need to be several enhancement releases. Producing a generally enhanced 2.8, 2.9, 2.10, etc, for the next decade would solve both problems -- but we are adamantly not going to do that. Instead, I think the PEP should propose a special series of server enhancement releases that are based on the final 2.7 maintenance release (2.7.8 or 2.7.9) but which have have a different application-specific enhancement policy. It would start with something like 'ServerPython 2.7.10', with the numbers moving upward from 10. I think one justification for this is that if PSF does not sanction such a series, it will happen anyway, but possibly with different and incompatible releases from different vendors or distribution families. As I understand the proposal, a subset of core developers would set the specification while some commercial group would have to fund it. The download page for the final 2.7.z maintenance release could say something like We recommend that you move to the most recent Python 3 version if at all possible. If you cannot do that and you want to use Python to run a server on the public internet, we urge you to instead use the latest version of ServerPython 2.7.1s. This series is based on Python 2.7.z but has been and will continue to be enhanced with security features backported from Python 3. Part of the idea above is that 2.7 development would end or mostly end after 5 years, the difference being that the last 3 years have not been restricted only to security bugs. Since security bugs are nearly always server problems, they would be fixed thereafter in ServerPython. If ServerPython is kept in the CPython repository, it would be a separate branch. I would, of course, also need enhancements needed to build sensibly on current systems. On Windows, for instance, it would be built with a current compiler. There are still details to be filled in or altered, but this is my general idea. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
