Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 25 March 2014 12:25, MRAB pyt...@mrabarnett.plus.com wrote: On 2014-03-25 01:29, Ben Darnell wrote: On Mon, Mar 24, 2014 at 4:44 AM, Nick Coghlan ncogh...@gmail.com mailto:ncogh...@gmail.com wrote: On 24 Mar 2014 15:25, Chris Angelico ros...@gmail.com mailto:ros...@gmail.com wrote: As has already been pointed out, this can already happen, but in an ad-hoc way. Making it official or semi-official would mean that a script written for Debian's Python 2.7.10 would run on Red Hat's Python 2.7.10, which would surely be an advantage. And having it break on the official Windows and Mac OS X binaries would benefit end users, how? The position I am coming to is that the enhanced security release should be the default one that we publish binary installers for, but we should also ensure that downstream redistributors can easily do Python 2.7 with legacy SSL releases if they so choose. I'm happier forcing end users to rely on a redistributor to opt in to a lower security option than I am to knowingly publish too many additional releases with network security infrastructure that is (at best) rapidly approaching its use by date. I am strongly against allowing downstream distributors that choice. Unless the security-enhanced variant of Python 2.7 quickly and completely overtakes all previous versions, we will be creating a compatibility problem between the two variants of Python 2.7. I believe that the changes motivating this PEP can be made with minimal backwards-incompatibility risk and (if the PEP is accepted) we should use all the leverage at our disposal to drive adoption. The risk is not backwards incompatibility, it is ambiguity of what Python 2.7 means. If changes under this PEP would result in any distributors rationally remaining at Python 2.7.6, then the result of any such changes should be labelled Python 2.8. I think that calling it Python 2.8 would be a bad idea for the reasons that have already been stated. Perhaps it should just be called Python 2.7 Enhanced Security (Python 2.7 ES). The PEP currently calls the proposed unmodified fork of 2.7 Python 2.7 with Legacy SSL. I suspect we could potentially ask the PSF to enforce that from a trademark perspective (that is, redistributors wouldn't be allowed to call versions with the legacy infrastructure Python 2.7, they'd have to include the with Legacy SSL qualifier - that would also encompass all redistributions of 2.7.6 and below). I'm actually personally OK with just making vendors do all the work if they're really so worried about a slightly increased chance of undetected regressions that they prefer to keep using older SSL infrastructure. I think persisting with the old SSL infrastructure for too much longer would be a fundamentally bad idea, so I don't mind at all making it more difficult for downstream redistributors to do so. As Ben notes, allowing them this option increases the chance of confusion about what Python 2.7 means, and once an upgrade Python 2.7 release was published, the Python 2.7 with Legacy SSL moniker would apply just as well to Python 2.7.6 and earlier as it would to a hypothetical additional branch that would impose an ongoing maintenance burden upstream. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 2): Network security enhancements for Python 2.7
On 25 March 2014 09:04, Donald Stufft don...@stufft.io wrote: On Mar 24, 2014, at 5:38 PM, Nick Coghlan ncogh...@gmail.com wrote: While I totally agree that it would be incredibly awesome if more companies put dedicated time into developing and maintaining CPython I don't think pushing all the blame on to them is accurate. The attitude towards security issues and backwards compatibility has a somewhat equal share in the causes of the aging security infrastructure of the 2.x line. Now this PEP, if accepted, does a lot to resolve the largest offenders of this policy (and there has been some signs lately that perhaps going forward this will be better) but I think it is not doing anyone a favor if we just point fingers *over there* and claim the fault lies with someone else doing or not doing something. I *don't* want to disparage anyone or anything of that like, mostly to say that while of course increased resources from corporate users would help the situation immensely but that additionally there is a reasonably sized contingent of influential members who still want to treat Python as a hobbyist project and not a critical piece of the infrastructure of the Internet as a whole. I *don't* want to get help from downstream users, especially on important but boring or hard issues such as security, and then have them feel shutdown and unable to actually get anything done as others who have attempted to resolve some of these issues in the past have had happen to them. I actually agree with this (hence why I wrote the PEP in the first place), I just became really, really, really, annoyed with certain organisations over the course of writing the PEP drafts and that is reflected in the tone of the latest draft. However, in deliberately not naming names, I now realise I've left it open to *other* organisations thinking Does he mean us? How is this our fault?. For clarification: if an org is guessing whether or not I was referring to them in particular while drafting the PEP, then no, I'm not. The specific organisations concerned are in absolutely no doubt as to the fact I'm genuinely angry with them. That said, while it certainly made me feel better at the time, I agree some of the current phrasing is not actually helpful in resolving the situation amicably for the benefit of all concerned, so I'll revise the offending sections of the PEP :) Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Am 25.03.2014 08:51, schrieb Nick Coghlan: I think that calling it Python 2.8 would be a bad idea for the reasons that have already been stated. Perhaps it should just be called Python 2.7 Enhanced Security (Python 2.7 ES). The PEP currently calls the proposed unmodified fork of 2.7 Python 2.7 with Legacy SSL. I suspect we could potentially ask the PSF to enforce that from a trademark perspective (that is, redistributors wouldn't be allowed to call versions with the legacy infrastructure Python 2.7, they'd have to include the with Legacy SSL qualifier - that would also encompass all redistributions of 2.7.6 and below). I don't know. It still feels like a source of confusion all round to have two different (C)Pythons not distinguished by version number. I haven't followed all of this thread, so forgive me if this suggestion has come up already: Since we know the EOL of 2.7, can't we say there won't be any more non-secure bugfix releases than up to 2.7.9, and the namespace 2.7.10 (yeah I know, but still way better than 2.8) and above is free for the new SSL versions. This also works from a version requirement point of view: if you require Python = 2.7.10 you know you'll get the new features. If you don't, you shouldn't be using (or carefully checking) the new opt-in features. I'm actually personally OK with just making vendors do all the work if they're really so worried about a slightly increased chance of undetected regressions that they prefer to keep using older SSL infrastructure. I think persisting with the old SSL infrastructure for too much longer would be a fundamentally bad idea, so I don't mind at all making it more difficult for downstream redistributors to do so. I agree, if no other solution can be found we should err on the secure side (as opposed to the safe side). Georg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Tue, Mar 25, 2014 at 7:12 PM, Cory Benfield c...@lukasa.co.uk wrote: On 24 March 2014 19:37, Chris Angelico ros...@gmail.com wrote: The opting in could be done at the distro level. Red Hat could ship a thing called /usr/bin/python that runs SEPython, and that package could be identified and numbered in such a way that it's clearly a drop-in replacement for vanilla Python. If backward compatibility is done carefully (which, from everything I'm seeing here, is the way it's to be done), there should be absolutely no downside to using SEPython, except for portability (because now you're depending on it for security), and corner cases like testing. What's your solution for OS X, Windows et al? My concern is that if you have a release called 'Python' and a release called 'Python with security stuff', a surprisingly large number of people will download the first, especially if the notes for the security release say 'this may cause some minor compatibility problems'. IMHO, I'd rather have good security be the default for everyone, and require an explicit opt-out to get the bad security release. Exactly the same. If someone wants to distribute SEPython (that someone might be python.org itself, or ActiveState, or anyone else who has an interest in it), they're welcome to do so; and it could be done either as an all-in-one that packages all of CPython, or as an add-on; either way would work just as well, but the former would be cleaner. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFE 20469: ssl.getpeercert() should include extensions
On 24.03.2014 23:51, Andrew M. Hettinger wrote: I thought I'd wait until the 3.4 release before I bothered asking about this: http://bugs.python.org/issue20469 I don't think I'm qualified to actually be writing code for the ssl module, but is there anything else that I can do to help? I could probably put together a demonstration-case if that would be helpful. I already have some code for the task. It wasn't ready for 3.4 but I'm going to add it to 3.5. I'm planing to include a X.509 type in order to wrap X.509 certificates. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Tue, Mar 25, 2014 at 7:37 PM, Cory Benfield c...@lukasa.co.uk wrote: On 25 March 2014 08:26, Chris Angelico ros...@gmail.com wrote: Exactly the same. If someone wants to distribute SEPython (that someone might be python.org itself, or ActiveState, or anyone else who has an interest in it), they're welcome to do so; and it could be done either as an all-in-one that packages all of CPython, or as an add-on; either way would work just as well, but the former would be cleaner. Reading this I suspect we're mostly in agreement but having trouble communicating. My understanding of your point is simply that you don't want python-dev to 'bless' either of the 2.7 releases proposed as _the_ 2.7 release, instead pushing that choice on to people distributing Python. I can get behind that plan so long as the source code releases are named in such a way that people are either a) forced to make a choice; or b) defaulted to secure 2.7. I'd like python.org / python-dev to bless, if not some specific version, at least some specific structure. I think that's something like what has been in the PEP at some point, though I haven't dug into the current version deeply enough to be sure. But if you take current 2.7 as a baseline, every new feature would be implemented by creating a new attribute of either the ssl module or some class in it; if the attribute is there, you can use it (eg a constant/enum value that's a parameter to something else), and if it's not, you can't. As long as the names are consistent, it'd be easy for a program to either probe and use what it can get, or just use what it wants and bomb if you don't give it a sufficiently-secure Python. So by that model, current 2.7 is fully compliant, and anything that doesn't actively conflict with that is also compliant. Any script that is written for the current 2.7 is guaranteed also to run on any compliant SEPython; and anything written for SEPython has to gracefully handle (which might mean cleanly bombing) anything down to and including current 2.7. Does that make sense? ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.4 buildbots available
Am 22.03.14 22:03, schrieb Benjamin Peterson: On Sat, Mar 22, 2014, at 11:10, Antoine Pitrou wrote: Hello, I've created the 3.4 category in the buildbots setup: http://buildbot.python.org/all/waterfall?category=3.4.stable I've also retired 3.2 and 3.3 buildbots. Someone will have to update the text and URLs at https://www.python.org/dev/buildbot/. We probably want a http://buildbot.python.org/3.4.stable/ redirect, too, then. Done! Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 2): Network security enhancements for Python 2.7
On Mar 25, 2014, at 06:11 PM, Nick Coghlan wrote: I actually agree with this (hence why I wrote the PEP in the first place), I just became really, really, really, annoyed with certain organisations over the course of writing the PEP drafts and that is reflected in the tone of the latest draft. However, in deliberately not naming names, I now realise I've left it open to *other* organisations thinking Does he mean us? How is this our fault?. For clarification: if an org is guessing whether or not I was referring to them in particular while drafting the PEP, then no, I'm not. The specific organisations concerned are in absolutely no doubt as to the fact I'm genuinely angry with them. That said, while it certainly made me feel better at the time, I agree some of the current phrasing is not actually helpful in resolving the situation amicably for the benefit of all concerned, so I'll revise the offending sections of the PEP :) Anger management through PEP writing! That's novel, but I can show you some more effective techniques at Pycon. :) -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On 25 March 2014 13:09, Nick Coghlan ncogh...@gmail.com wrote: * MvL has indicated he is not prepared to tackle the task of trying to integrate a newer OpenSSL into the also aging Python 2.7 build infrastructure on Windows (unfortunately, we've looked into upgrading that build infrastructure, and the backwards compatibility issues appear to be effectively insurmountable). We would require a commitment from another trusted contributor to handle at least this task, and potentially also taking over the task of creating the official Python 2.7 Windows installers for the remaining Python 2.7 maintenance releases. One issue that strikes me is that much of the focus of this PEP is on supporting Linux distributions. This is entirely reasonable, as they are the ones with the sort of long-term support commitments that result in this issue (in the Windows world, possibly ActiveState offer formal support for Python 2.7, but otherwise I'm not aware of actual paid support options that might be relevant on Windows). With that in mind, is it reasonable to expect Linux vendors to support delivery of updated Windows builds of Python 2.7? If not, is it acceptable to python-dev to release a Python 2.7 maintenance release with backported security enhancements only available for Linux? (The same questions can be asked of OSX or Solaris support - this isn't solely a Windows issue). I think the PEP needs to be explicit here about what python-dev expect in terms of cross-platform support. I would assume that the expectation is that we deliver exactly the same level of cross-platform support as for 3.x, but commercial vendors could quite easily miss that implication if it is not spelled out. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On 25 Mar 2014 23:29, Paul Moore p.f.mo...@gmail.com wrote: On 25 March 2014 13:09, Nick Coghlan ncogh...@gmail.com wrote: * MvL has indicated he is not prepared to tackle the task of trying to integrate a newer OpenSSL into the also aging Python 2.7 build infrastructure on Windows (unfortunately, we've looked into upgrading that build infrastructure, and the backwards compatibility issues appear to be effectively insurmountable). We would require a commitment from another trusted contributor to handle at least this task, and potentially also taking over the task of creating the official Python 2.7 Windows installers for the remaining Python 2.7 maintenance releases. One issue that strikes me is that much of the focus of this PEP is on supporting Linux distributions. This is entirely reasonable, as they are the ones with the sort of long-term support commitments that result in this issue (in the Windows world, possibly ActiveState offer formal support for Python 2.7, but otherwise I'm not aware of actual paid support options that might be relevant on Windows). With that in mind, is it reasonable to expect Linux vendors to support delivery of updated Windows builds of Python 2.7? If not, is it acceptable to python-dev to release a Python 2.7 maintenance release with backported security enhancements only available for Linux? (The same questions can be asked of OSX or Solaris support - this isn't solely a Windows issue). I think the PEP needs to be explicit here about what python-dev expect in terms of cross-platform support. I would assume that the expectation is that we deliver exactly the same level of cross-platform support as for 3.x, but commercial vendors could quite easily miss that implication if it is not spelled out. The PEP says to sync with Python 3, and that has full cross platform support. The Linux focus just comes from the fact that Linux is where the problem is most evident. It's not like we're going to just be giving the PEP to vendors as a spec and leaving them to it - it's largely an invitation to participate more directly upstream to help resolve a particularly thorny problem, not a Statement of Work :) Cheers, Nick. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On 25 March 2014 13:47, Nick Coghlan ncogh...@gmail.com wrote: It's not like we're going to just be giving the PEP to vendors as a spec and leaving them to it - it's largely an invitation to participate more directly upstream to help resolve a particularly thorny problem, not a Statement of Work :) :-) I don't really know the APIs involved, but AIUI one of the 3.4 enhancements is exposing the SSLContext. Is the code to do this compatible with the version of OpenSSL bundled with Python 2.7 on Windows? If not, suppose that Red Hat provide resources that work on backporting the code, but they don't have Windows experts so no-one deals with integrating the new OpenSSL into the Windows binaries. Would the backport be blocked until someone is found to do the Windows work? This I've written a patch but it hasn't been applied is the type of scenario that puts people off contributing. If it's likely to happen, I think Red Hat have a right to know that in advance. And I don't know that it's something they would appreciate without python-dev pointing it out. If we're reasonably sure (not necessarily certain, there's always grey areas) that this isn't going to be an issue, then that's also fine. We can simply say that. That's all I'm saying. Not trying to require anything of contributors, just trying to be open and explicit about the criteria that will apply to accepting contributions. Paul. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Tue, 25 Mar 2014 14:25:29 + Paul Moore p.f.mo...@gmail.com wrote: On 25 March 2014 13:47, Nick Coghlan ncogh...@gmail.com wrote: It's not like we're going to just be giving the PEP to vendors as a spec and leaving them to it - it's largely an invitation to participate more directly upstream to help resolve a particularly thorny problem, not a Statement of Work :) :-) I don't really know the APIs involved, but AIUI one of the 3.4 enhancements is exposing the SSLContext. No, it was done as soon as 3.2. Note that exposing SSLContext doesn't achieve anything *in itself*, until you also add relevant APIs to query and influence the SSLContext's internal state (which was done gradually in 3.2, 3.3 and 3.4). Separate enhancements have to do with strengthening the default SSL options and accepted SSL ciphers; when they didn't add new APIs or threatened to break legitimate use case, they have even been applied to bugfix branches. Is the code to do this compatible with the version of OpenSSL bundled with Python 2.7 on Windows? It is. Actually, we have (or had) buildbots with older OpenSSL versions! The problem with backporting SSLContext is really that 1) you are adding a non-minimal new API set to the 2.7 feature set; 2) you must make it so that other stdlib modules take advantage of the new APIs (otherwise why bother?). This I've written a patch but it hasn't been applied is the type of scenario that puts people off contributing. In the case of ssl, I think I've tried to apply most interesting and finished patches. This is how you get features such as NPN protocols or server-side SNI. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 25 March 2014 08:26, Chris Angelico ros...@gmail.com wrote: Exactly the same. If someone wants to distribute SEPython (that someone might be python.org itself, or ActiveState, or anyone else who has an interest in it), they're welcome to do so; and it could be done either as an all-in-one that packages all of CPython, or as an add-on; either way would work just as well, but the former would be cleaner. Reading this I suspect we're mostly in agreement but having trouble communicating. My understanding of your point is simply that you don't want python-dev to 'bless' either of the 2.7 releases proposed as _the_ 2.7 release, instead pushing that choice on to people distributing Python. I can get behind that plan so long as the source code releases are named in such a way that people are either a) forced to make a choice; or b) defaulted to secure 2.7. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 24 March 2014 19:37, Chris Angelico ros...@gmail.com wrote: The opting in could be done at the distro level. Red Hat could ship a thing called /usr/bin/python that runs SEPython, and that package could be identified and numbered in such a way that it's clearly a drop-in replacement for vanilla Python. If backward compatibility is done carefully (which, from everything I'm seeing here, is the way it's to be done), there should be absolutely no downside to using SEPython, except for portability (because now you're depending on it for security), and corner cases like testing. What's your solution for OS X, Windows et al? My concern is that if you have a release called 'Python' and a release called 'Python with security stuff', a surprisingly large number of people will download the first, especially if the notes for the security release say 'this may cause some minor compatibility problems'. IMHO, I'd rather have good security be the default for everyone, and require an explicit opt-out to get the bad security release. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 25 March 2014 09:01, Chris Angelico ros...@gmail.com wrote: So by that model, current 2.7 is fully compliant, and anything that doesn't actively conflict with that is also compliant. Any script that is written for the current 2.7 is guaranteed also to run on any compliant SEPython; and anything written for SEPython has to gracefully handle (which might mean cleanly bombing) anything down to and including current 2.7. Does that make sense? Absolutely. =) My additional concern on top of that is wanting users to fall into a pit of success by making it overwhelmingly more likely that users will accidentally end up with the safe version if they aren't paying attention. I'm not hugely bothered about how that's done: I'd just like not to have to field Requests bug reports about lack of security that boil down to a user having grabbed the insecure version by accident. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Tue Mar 25 2014 at 4:21:51 AM, Georg Brandl g.bra...@gmx.net wrote: Am 25.03.2014 08:51, schrieb Nick Coghlan: I think that calling it Python 2.8 would be a bad idea for the reasons that have already been stated. Perhaps it should just be called Python 2.7 Enhanced Security (Python 2.7 ES). The PEP currently calls the proposed unmodified fork of 2.7 Python 2.7 with Legacy SSL. I suspect we could potentially ask the PSF to enforce that from a trademark perspective (that is, redistributors wouldn't be allowed to call versions with the legacy infrastructure Python 2.7, they'd have to include the with Legacy SSL qualifier - that would also encompass all redistributions of 2.7.6 and below). I don't know. It still feels like a source of confusion all round to have two different (C)Pythons not distinguished by version number. I haven't followed all of this thread, so forgive me if this suggestion has come up already: Since we know the EOL of 2.7, can't we say there won't be any more non-secure bugfix releases than up to 2.7.9, and the namespace 2.7.10 (yeah I know, but still way better than 2.8) and above is free for the new SSL versions. This also works from a version requirement point of view: if you require Python = 2.7.10 you know you'll get the new features. If you don't, you shouldn't be using (or carefully checking) the new opt-in features. Or if this is such a big deal we start with 2.7.6 and not postpone until 2.7.10 (which I guess could happen immediately after 2.7.9 and have nothing more than the upgraded modules). People have been making grandiose statements about how the security of the internet is hampered by Python 2.7 in this discussion. If these statements are actually not over-stated then we should do the fix sooner *and* add the incentive people to switch over by getting more bug fixes. It's not like Python 2.7 is getting a ton of fixes at this point anyway. I'm actually personally OK with just making vendors do all the work if they're really so worried about a slightly increased chance of undetected regressions that they prefer to keep using older SSL infrastructure. I think persisting with the old SSL infrastructure for too much longer would be a fundamentally bad idea, so I don't mind at all making it more difficult for downstream redistributors to do so. I agree, if no other solution can be found we should err on the secure side (as opposed to the safe side). As long as we make it clear we have chosen to change our backwards-compatibility guarantees in the name of security and have a link to the last backwards-compatible release then I agree as well. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Tue, Mar 25, 2014 at 8:10 AM, Antoine Pitrou solip...@pitrou.net wrote: The problem with backporting SSLContext is really that 1) you are adding a non-minimal new API set to the 2.7 feature set; 2) you must make it so that other stdlib modules take advantage of the new APIs (otherwise why bother?). I actually worry about another scenario. Suppose we backport SSLContext to 2.7.7. There is plenty of software out there nowadays that uses a single source for Python 2 and 3 compatibility. Given that SSLContext until now has only been available in Python 3, and given the recommendation to use dynamic feature detection, not version checks, it's at least *conceivable* that some library detects SSLContext and switches to some code that uses Python-3-only features (of the variety that don't cause syntax errors in Python 2). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
A casual glance at https://github.com/kennethreitz/requests/blob/master/requests/packages/urllib3/ util.py#L610 which is probably the most widely used consumer of these APIs, outside the stdlib itself, looks to me like if these names were to suddenly show up, everything would continue to work just fine, with the advance of being able to explicitly specify some options. All of which is to say: I don't think this is a real concern. Alex ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Tue, Mar 25, 2014 at 8:31 AM, Alex Gaynor alex.gay...@gmail.com wrote: A casual glance at https://github.com/kennethreitz/requests/blob/master/requests/packages/urllib3/ util.py#L610 which is probably the most widely used consumer of these APIs, outside the stdlib itself, looks to me like if these names were to suddenly show up, everything would continue to work just fine, with the advance of being able to explicitly specify some options. All of which is to say: I don't think this is a real concern. That would be great, because I have no other major beef with the PEP (but I still need to read in in full -- it's long and half of it still feels like weasel words to me, so I can't apply my usual skimming tactics). I would like the PEP (or perhaps a companion PEP?) spell out the set of enhancements that we would *currently* like to see backported from Python 3.4 to 2.7, without the implication that these would be the *only* enhancements -- such a list would serve as an example and to focus the understanding. The PEP currently doesn't even name SSLContext! I wouldn't be totally surprised to find that there are some details of some API added to Python 3.4 that simply cannot be backported due to some important difference between Python 2 and 3 (e.g. because of differences in Unicode handling, or a missing socket method). I don't think such things would be showstoppers, they would just have to be worked around carefully; but it would be better to know about them now rather than having to figure out how to comply with the PEP's insistence of a full backport. I do note that the PEP seems to have some weasel-words about breaking backward compatibility in the name of security. The phrase This PEP does *not* grant Python 2.7 any general exemptions to the usual backwards compatibility policy for maintenance releases *could* be interpreted to imply that the PEP grants some *specific* exemptions (regardless of whether that was Nick's intention when he wrote that sentence). I'd like clarity on this; IIRC we've had to make some compatibility-breaking changes in the past for security reasons, but I don't recall the details or how that worked out (whether much code broke and whether that was considered a good or a bad thing). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Mar 25, 2014, at 12:35 PM, Guido van Rossum gu...@python.org wrote: On Tue, Mar 25, 2014 at 8:31 AM, Alex Gaynor alex.gay...@gmail.com wrote: A casual glance at https://github.com/kennethreitz/requests/blob/master/requests/packages/urllib3/ util.py#L610 which is probably the most widely used consumer of these APIs, outside the stdlib itself, looks to me like if these names were to suddenly show up, everything would continue to work just fine, with the advance of being able to explicitly specify some options. All of which is to say: I don't think this is a real concern. That would be great, because I have no other major beef with the PEP (but I still need to read in in full -- it's long and half of it still feels like weasel words to me, so I can't apply my usual skimming tactics). I would like the PEP (or perhaps a companion PEP?) spell out the set of enhancements that we would *currently* like to see backported from Python 3.4 to 2.7, without the implication that these would be the *only* enhancements -- such a list would serve as an example and to focus the understanding. The PEP currently doesn't even name SSLContext! I wouldn't be totally surprised to find that there are some details of some API added to Python 3.4 that simply cannot be backported due to some important difference between Python 2 and 3 (e.g. because of differences in Unicode handling, or a missing socket method). I don't think such things would be showstoppers, they would just have to be worked around carefully; but it would be better to know about them now rather than having to figure out how to comply with the PEP's insistence of a full backport. I do note that the PEP seems to have some weasel-words about breaking backward compatibility in the name of security. The phrase This PEP does not grant Python 2.7 any general exemptions to the usual backwards compatibility policy for maintenance releases *could* be interpreted to imply that the PEP grants some specific exemptions (regardless of whether that was Nick's intention when he wrote that sentence). I'd like clarity on this; IIRC we've had to make some compatibility-breaking changes in the past for security reasons, but I don't recall the details or how that worked out (whether much code broke and whether that was considered a good or a bad thing). I’m pretty sure Nick was just trying to say that the changes made under this PEP still have to be backwards compatible in the sense that APIs can’t change their default behavior and such. In other words we can’t suddenly flip on hostname checking or anything like that. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.3 branch is now in security fix mode
Sent from my iPhone ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Tue, Mar 25, 2014 at 9:46 AM, Donald Stufft don...@stufft.io wrote: On Mar 25, 2014, at 12:35 PM, Guido van Rossum gu...@python.org wrote: [...] I do note that the PEP seems to have some weasel-words about breaking backward compatibility in the name of security. The phrase This PEP does *not* grant Python 2.7 any general exemptions to the usual backwards compatibility policy for maintenance releases *could* be interpreted to imply that the PEP grants some *specific* exemptions (regardless of whether that was Nick's intention when he wrote that sentence). I'd like clarity on this; IIRC we've had to make some compatibility-breaking changes in the past for security reasons, but I don't recall the details or how that worked out (whether much code broke and whether that was considered a good or a bad thing). I'm pretty sure Nick was just trying to say that the changes made under this PEP still have to be backwards compatible in the sense that APIs can't change their default behavior and such. In other words we can't suddenly flip on hostname checking or anything like that. Then the words should be clarified (maybe by removing 'general'?). This PEP invites interpretation by future generations so it should be as clear as possible on the intent, to avoid scholarly arguments. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Mar 25, 2014, at 1:16 PM, Guido van Rossum gu...@python.org wrote: On Tue, Mar 25, 2014 at 9:46 AM, Donald Stufft don...@stufft.io wrote: On Mar 25, 2014, at 12:35 PM, Guido van Rossum gu...@python.org wrote: [...] I do note that the PEP seems to have some weasel-words about breaking backward compatibility in the name of security. The phrase This PEP does not grant Python 2.7 any general exemptions to the usual backwards compatibility policy for maintenance releases *could* be interpreted to imply that the PEP grants some specific exemptions (regardless of whether that was Nick's intention when he wrote that sentence). I'd like clarity on this; IIRC we've had to make some compatibility-breaking changes in the past for security reasons, but I don't recall the details or how that worked out (whether much code broke and whether that was considered a good or a bad thing). I’m pretty sure Nick was just trying to say that the changes made under this PEP still have to be backwards compatible in the sense that APIs can’t change their default behavior and such. In other words we can’t suddenly flip on hostname checking or anything like that. Then the words should be clarified (maybe by removing 'general'?). This PEP invites interpretation by future generations so it should be as clear as possible on the intent, to avoid scholarly arguments. -- --Guido van Rossum (python.org/~guido) Yea I agree, was just stating what I understand the PEP to be proposing :) - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
Am 25.03.14 14:47, schrieb Nick Coghlan: The PEP says to sync with Python 3, and that has full cross platform support. The Linux focus just comes from the fact that Linux is where the problem is most evident. However, it fails to address a critical detail: the upcoming maintenance end for 2.7. This makes it less platform-agnostic than you say: For Linux, many users rely on the system vendor to provide Python binary, and they may chose to continue improving the backported APIs even after python-dev's maintenance ends. OTOH, for Windows, most users rely on the binaries from python.org, which will stop evolving in May 2015. Of course, there is still ActiveState, but there isn't the same kind of vendor choice that you see for Linux. So I think the PEP should make it clear whether: a) you expect that the maintenance period for Python 2.7 is to be extended beyond 2015, b) you expect that the backports will evolve in the years to come, or whether this will be a one-shot activity c) and if the answer to b was will evolve, whether you expect this will create forks of 2.7 after maintenance has ended (as the then-security-only 2.7 branch won't see any further improvement to the backport, according to the PEP) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
Nick Coghlan writes: Changes to these modules will still need to undergo normal backwards compatibility assessments to ensure their default behaviour remains consistent with earlier Python 2.7 releases, I find this wording confusing. Does this mean that (possibly imaginary example) where Python 3.n enables a new host match check by default that Python 2.7 doesn't have, Python 2.7 gets the ability to check host match, but it defaults to off? While this PEP does not make any changes to the core development team's handling of security-fix-only branches that are no longer in active maintenance, I wonder if it would be possible to generalize the approach to 2.7 in such a way that our repos can collect backports of security *enhancements* which we have reviewed, even though they wouldn't be distributed in a release? That way the distros can share their patches to 2.x, x 7. infrastructure on their platform. Accordingly, and independently of this PEP, the Mac OS X binary installers were already going to be switched to statically linker newer versions of OpenSSL [4]_ Typo: statically linker - statically-linked For better or for worse (mostly worse), there are some environments where Is that For better or for worse (mostly worse) really necessary? Consenting adults and all that. the risk of latent security defects is more tolerated than even a slightly increased risk of regressions in maintenance releases. This policy largely excludes these environments from consideration where the modules covered by the exemption are concerned Again, I think you can put a period here. The people making such decisions know what they're doing, at least they think they do. Just tell them they're on they're own now, and their complaints, if any, will fall on deaf ears. Downstream redistributors may still choose to cater to such environments, but they will need to handle the process of downgrading the security related modules and doing the associated regression testing themselves. The main CPython continuous integration infrastructure will not cover this scenario. This is how to write it, I think. Motivation and Rationale This PEP can be seen as a more targeted version of the faster standard library release cycle proposals discussed in PEP 407 and PEP 413, focusing specifically on those areas which have implications beyond the Python community. FWIW, I don't see it that way at all. It is worth comparing the approach described in this PEP with Red Hat's handling of its long term support commitments: it isn't the RHEL 6.0 release itself that receives 10 years worth of support, but the overall RHEL 6 *series*. The individual RHEL 6.x point releases within the series then receive a wide variety of new features, including security enhancements, all while meeting strict backwards compatibility guarantees for existing software. I don't understand this comparison. I think it requires too much familiarity with Red Hat policy to be useful to the great majority of readers, ie, it tells them a lot more about Red Hat then it does about Python. However, now that we're fully aware of the impact the limitations of the Python 2 standard library may be having on the evolution of internet security standards, I suggest s/evolution/implementation/ here. As Terry Reedy noted, URL? Alternative: create and release Python 2.8 -- I don't see how this can possibly be viable. It would have to involve essentially the same restrictions as this PEP, or it completely fails to satisfy the requirements of downstreams. Even then, it would be a huge, uphill PR battle for downstreams trying to enhance their current 2.7 with the content of this PEP. With sufficient corporate support, it likely *would* be possible to create and release Python 2.8 (it's highly unlikely such a project would garner enough interest to be achievable with only volunteers). However, this wouldn't actually solve the problem, as the aim is to provide a *relatively low impact* way to incorporate enhanced security features into integrated products and deployments that make use of Python 2. Upgrading to a new Python feature release would mean both more work for the core development team, as well as a more disruptive update that most potential end users would likely just skip entirely. Attempting to create a Python 2.8 release would also bring in suggestions to backport many additional features from Python 3 (such as ``tracemalloc`` and the improved coroutine support), making the migration from Python 2.7 to this hypothetical 2.8 release even riskier and more disruptive. This is not a recommended approach, as it would involve substantial additional work for a result that is actually less effective in achieving the original aim (which is to eliminate the current widespread use of the
[Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
Urk. Premature send. As it turns out, I had no specific comments to make after the one comment on Alternative: create and release Python 2.8 -- My apologies to anybody who read to the bottom for wasting their time. I do have one generic comment on style: I think at this point the statements tagged with I (Nick Coghlan) think should either be edited to remove the reference to the author, or moved to the Caveat: I work for RH section or the Open Issues section. If they're not pretty much consensus by now, they probably don't belong in the main body of the PEP. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 26 Mar 2014 01:19, Brett Cannon bcan...@gmail.com wrote: As long as we make it clear we have chosen to change our backwards-compatibility guarantees in the name of security and have a link to the last backwards-compatible release then I agree as well. I am not sure how this meme got started, but let me be clear: the proposed policy DOES NOT provide blanket permission to break backwards compatibility in the affected modules. It only allows ADDING new features to bring these modules into line with their Python 3 counterparts, making it easier for third party packages like requests to do the right thing in a cross-version compatible way. The use feature detection, not version checks guidelines in the PEP are designed to deal with the concerns around subsequently missing features in the existing Python 2.7 releases. The remaining concern appears to be largely around the slightly increased chance of regressions that comes with making larger changes to these modules in order to incorporate the new features. Given our regression test suite, and those of other projects like OpenStack and components of the Linux distributions, I now consider that concern to be entirely misplaced. The only backwards compatibility breaks allowed general exemption applies to the new ssl.create_default_context() function, which *is* defined as allowing backwards incompatible changes to keep up with evolving security requirements. That exemption is in the documentation of that API, though - the only impact of this PEP would be to also make that API available in 2.7.7+. Regards, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Status of PEP 3145 - Asynchronous I/O for subprocess.popen
Hi, On core-mentorship someone asked about PEP 3145 - Asynchronous I/O for subprocess.popen. I answered that asyncio now has subprocess support (including non-blocking I/O on the three standard stream pipes), so it's not obvious anything else is needed. Should we change the PEP's status to Rejected or Superseded? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Am 25.03.2014 23:15, schrieb Nick Coghlan: On 26 Mar 2014 01:19, Brett Cannon bcan...@gmail.com mailto:bcan...@gmail.com wrote: As long as we make it clear we have chosen to change our backwards-compatibility guarantees in the name of security and have a link to the last backwards-compatible release then I agree as well. I am not sure how this meme got started, but let me be clear: the proposed policy DOES NOT provide blanket permission to break backwards compatibility in the affected modules. It only allows ADDING new features to bring these modules into line with their Python 3 counterparts, making it easier for third party packages like requests to do the right thing in a cross-version compatible way. We know. That's what we mean by that. Georg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Tue, 25 Mar 2014 23:09:45 +1000 Nick Coghlan ncogh...@gmail.com wrote: Alternative: selectively backport particular APIs - An instinctively minimalist reaction to this proposal is to only backport particular APIs in the affected modules that are judged to be security critical. However, this ends up providing a worse end user experience, as well as a worse developer experience. For end users, the selective backporting approach means learning not only the legacy Python 2.7 API and the current Python 3 APIs, but also the hybrid API created by the selective backporting process. I think this is a strawman, since you are also advocating for a feature detection approach to writing cross-version code. It is already required, actually, if wanting to write code compatible from 3.2 to 3.4 (for example, SSLContext exists in 3.2 but create_default_context appears in 3.4 while OP_NO_COMPRESSION appears in 3.3). I would much rather selectively backport a minimal set of APIs than the whole range of ssl APIs. There are things there (RAND_bytes, RAND_pseudo_bytes) that are not even useful for network security, or only in a rather uncommon manner (such as channel bindings). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On 26 Mar 2014 00:25, Paul Moore p.f.mo...@gmail.com wrote: On 25 March 2014 13:47, Nick Coghlan ncogh...@gmail.com wrote: It's not like we're going to just be giving the PEP to vendors as a spec and leaving them to it - it's largely an invitation to participate more directly upstream to help resolve a particularly thorny problem, not a Statement of Work :) :-) I don't really know the APIs involved, but AIUI one of the 3.4 enhancements is exposing the SSLContext. Is the code to do this compatible with the version of OpenSSL bundled with Python 2.7 on Windows? If not, suppose that Red Hat provide resources that work on backporting the code, but they don't have Windows experts so no-one deals with integrating the new OpenSSL into the Windows binaries. Would the backport be blocked until someone is found to do the Windows work? We'll get it done. For example, while *I* definitely approach the problem from a Linux vendor perspective (and that's reflected in the PEP), I also know several folks at Rackspace have expressed concern about the status quo, and the client side of OpenStack is cross platform. This I've written a patch but it hasn't been applied is the type of scenario that puts people off contributing. If it's likely to happen, I think Red Hat have a right to know that in advance. And I don't know that it's something they would appreciate without python-dev pointing it out. If we're reasonably sure (not necessarily certain, there's always grey areas) that this isn't going to be an issue, then that's also fine. We can simply say that. Yeah, I think we can make sure the right folks are involved to make it happen. The PEP is about me getting agreement in advance that we actually want to see the problem fixed, and the constraints we want to impose on the solution. Once we have that agreement, I won't be sitting around idly *waiting* for assistance to magically appear - I'll go looking for it, and I know there are others that will do the same :) That's all I'm saying. Not trying to require anything of contributors, just trying to be open and explicit about the criteria that will apply to accepting contributions. I won't expect Linux folks to fix Windows problems (as that rarely works well). If the PEP is accepted, I *will* ensure we get the policy implemented on all supported platforms for 2.7.7+ by getting appropriate people involved (and will also work on securing the appropriate longer term support commitments). Cheers, Nick. Paul. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On 26 Mar 2014 08:35, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 25 Mar 2014 23:09:45 +1000 Nick Coghlan ncogh...@gmail.com wrote: Alternative: selectively backport particular APIs - An instinctively minimalist reaction to this proposal is to only backport particular APIs in the affected modules that are judged to be security critical. However, this ends up providing a worse end user experience, as well as a worse developer experience. For end users, the selective backporting approach means learning not only the legacy Python 2.7 API and the current Python 3 APIs, but also the hybrid API created by the selective backporting process. I think this is a strawman, since you are also advocating for a feature detection approach to writing cross-version code. It is already required, actually, if wanting to write code compatible from 3.2 to 3.4 (for example, SSLContext exists in 3.2 but create_default_context appears in 3.4 while OP_NO_COMPRESSION appears in 3.3). I would much rather selectively backport a minimal set of APIs than the whole range of ssl APIs. There are things there (RAND_bytes, RAND_pseudo_bytes) that are not even useful for network security, or only in a rather uncommon manner (such as channel bindings). Yeah, I think this is a valid point, and, as Guido noted, we also want the option to skip backporting things if they depend on other aspects of Python 3 that we decide can't be backported. So a feature-by-feature decision making process actually does make more sense than a blanket exemption. Cheers, Nick. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 26 Mar 2014 08:32, Georg Brandl g.bra...@gmx.net wrote: Am 25.03.2014 23:15, schrieb Nick Coghlan: On 26 Mar 2014 01:19, Brett Cannon bcan...@gmail.com mailto:bcan...@gmail.com wrote: As long as we make it clear we have chosen to change our backwards-compatibility guarantees in the name of security and have a link to the last backwards-compatible release then I agree as well. I am not sure how this meme got started, but let me be clear: the proposed policy DOES NOT provide blanket permission to break backwards compatibility in the affected modules. It only allows ADDING new features to bring these modules into line with their Python 3 counterparts, making it easier for third party packages like requests to do the right thing in a cross-version compatible way. We know. That's what we mean by that. That's not what Brett said - he called 2.7.6 the last backwards compatible release. That's not correct, as even under my proposal, 2.7.7+ will still be backwards compatible. Cheers, Nick. Georg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of PEP 3145 - Asynchronous I/O for subprocess.popen
On 26 Mar 2014 08:22, Antoine Pitrou solip...@pitrou.net wrote: Hi, On core-mentorship someone asked about PEP 3145 - Asynchronous I/O for subprocess.popen. I answered that asyncio now has subprocess support (including non-blocking I/O on the three standard stream pipes), so it's not obvious anything else is needed. Should we change the PEP's status to Rejected or Superseded? Yes. I think we'd typically use Rejected in this case, as Superseded normally relates to the evolution of interface definition PEPs. Cheers, Nick. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of PEP 3145 - Asynchronous I/O for subprocess.popen
That would be a rather strong unilateral decision. Why don't you ask the authors? In theory the PEP's proposals could serve in situations where asyncio isn't appropriate, and asyncio's subprocess I/O isn't the smoothest API imaginable. (In practice I'm not sure if the PEP would have been written with asyncio subprocess support in place.) On Tue, Mar 25, 2014 at 3:19 PM, Antoine Pitrou solip...@pitrou.net wrote: Hi, On core-mentorship someone asked about PEP 3145 - Asynchronous I/O for subprocess.popen. I answered that asyncio now has subprocess support (including non-blocking I/O on the three standard stream pipes), so it's not obvious anything else is needed. Should we change the PEP's status to Rejected or Superseded? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Mar 25, 2014, at 7:03 PM, Nick Coghlan ncogh...@gmail.com wrote: On 26 Mar 2014 08:35, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 25 Mar 2014 23:09:45 +1000 Nick Coghlan ncogh...@gmail.com wrote: Alternative: selectively backport particular APIs - An instinctively minimalist reaction to this proposal is to only backport particular APIs in the affected modules that are judged to be security critical. However, this ends up providing a worse end user experience, as well as a worse developer experience. For end users, the selective backporting approach means learning not only the legacy Python 2.7 API and the current Python 3 APIs, but also the hybrid API created by the selective backporting process. I think this is a strawman, since you are also advocating for a feature detection approach to writing cross-version code. It is already required, actually, if wanting to write code compatible from 3.2 to 3.4 (for example, SSLContext exists in 3.2 but create_default_context appears in 3.4 while OP_NO_COMPRESSION appears in 3.3). I would much rather selectively backport a minimal set of APIs than the whole range of ssl APIs. There are things there (RAND_bytes, RAND_pseudo_bytes) that are not even useful for network security, or only in a rather uncommon manner (such as channel bindings). Yeah, I think this is a valid point, and, as Guido noted, we also want the option to skip backporting things if they depend on other aspects of Python 3 that we decide can't be backported. So a feature-by-feature decision making process actually does make more sense than a blanket exemption. Looking at the ssl module, just about the only thing that wouldn’t be helpful to have backported is the rand functions that AP mentioned (and those mostly because everyone should just use os.urandom for everything ever). The NPN stuff isn’t important for security related stuff either though it’d probably be more work to rip it out and more disruptive as well. Looking at the hmac module, the only thing that really matters for a backport would be the constant time compare. Lokoing at hashlib, the guaranteed and supported algorithms would be nice to have but not really security sensitive. Adding pbkdf2_hmac would be really nice for the security of web services though. Looking at os.urandom, back porting the lazily opened and held file descriptor would be a nice to have, but not strictly required. (Not sure if that would be considered a backwards incompat change). Nothing in random.py really stands out to me looking at it’s docs. Cheers, Nick. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of PEP 3145 - Asynchronous I/O for subprocess.popen
On Tue, 25 Mar 2014 16:14:04 -0700 Guido van Rossum gu...@python.org wrote: That would be a rather strong unilateral decision. Why don't you ask the authors? In theory the PEP's proposals could serve in situations where asyncio isn't appropriate, and asyncio's subprocess I/O isn't the smoothest API imaginable. (In practice I'm not sure if the PEP would have been written with asyncio subprocess support in place.) That's a good point. I now have e-mailed Eric Pruitt and Josiah Carlson (I couldn't find an e-mail for Charles R. McCreary). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 3
Okay, I included that last round of comments (from late February).
Barring typos, this should be the final version.
Final comments?
-
PEP: 461
Title: Adding % formatting to bytes and bytearray
Version: $Revision$
Last-Modified: $Date$
Author: Ethan Furman et...@stoneleaf.us
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 2014-01-13
Python-Version: 3.5
Post-History: 2014-01-14, 2014-01-15, 2014-01-17, 2014-02-22, 2014-03-25
Resolution:
Abstract
This PEP proposes adding % formatting operations similar to Python 2's ``str``
type to ``bytes`` and ``bytearray`` [1]_ [2]_.
Rationale
=
While interpolation is usually thought of as a string operation, there are
cases where interpolation on ``bytes`` or ``bytearrays`` make sense, and the
work needed to make up for this missing functionality detracts from the overall
readability of the code.
Motivation
==
With Python 3 and the split between ``str`` and ``bytes``, one small but
important area of programming became slightly more difficult, and much more
painful -- wire format protocols [3]_.
This area of programming is characterized by a mixture of binary data and
ASCII compatible segments of text (aka ASCII-encoded text). Bringing back a
restricted %-interpolation for ``bytes`` and ``bytearray`` will aid both in
writing new wire format code, and in porting Python 2 wire format code.
Common use-cases include ``dbf`` and ``pdf`` file formats, ``email``
formats, and ``FTP`` and ``HTTP`` communications, among many others.
Proposed semantics for ``bytes`` and ``bytearray`` formatting
=
%-interpolation
---
All the numeric formatting codes (such as ``%x``, ``%o``, ``%e``, ``%f``,
``%g``, etc.) will be supported, and will work as they do for str, including
the padding, justification and other related modifiers. The only difference
will be that the results from these codes will be ASCII-encoded text, not
unicode. In other words, for any numeric formatting code `%x`::
b%x % val
is equivalent to
(%x % val).encode(ascii)
Examples::
b'%4x' % 10
b' a'
b'%#4x' % 10
' 0xa'
b'%04X' % 10
'000A'
``%c`` will insert a single byte, either from an ``int`` in range(256), or from
a ``bytes`` argument of length 1, not from a ``str``.
Examples::
b'%c' % 48
b'0'
b'%c' % b'a'
b'a'
``%s`` is included for two reasons: 1) `b` is already a format code for
``format`` numerics (binary), and 2) it will make 2/3 code easier as Python 2.x
code uses ``%s``; however, it is restricted in what it will accept::
- input type supports ``Py_buffer`` [6]_?
use it to collect the necessary bytes
- input type is something else?
use its ``__bytes__`` method [7]_ ; if there isn't one, raise a
``TypeError``
In particular, ``%s`` will not accept numbers (use a numeric format code for
that), nor ``str`` (encode it to ``bytes``).
Examples::
b'%s' % b'abc'
b'abc'
b'%s' % 'some string'.encode('utf8')
b'some string'
b'%s' % 3.14
Traceback (most recent call last):
...
TypeError: b'%s' does not accept numbers, use a numeric code instead
b'%s' % 'hello world!'
Traceback (most recent call last):
...
TypeError: b'%s' does not accept 'str', it must be encoded to `bytes`
``%a`` will call ``ascii()`` on the interpolated value. This is intended
as a debugging aid, rather than something that should be used in production.
Non-ASCII values will be encoded to either ``\xnn`` or ``\u``
representation. Use cases include developing a new protocol and writing
landmarks into the stream; debugging data going into an existing protocol
to see if the problem is the protocol itself or bad data; a fall-back for a
serialization format; or even a rudimentary serialization format when
defining ``__bytes__`` would not be appropriate [8].
.. note::
If a ``str`` is passed into ``%a``, it will be surrounded by quotes.
Unsupported codes
-
``%r`` (which calls ``__repr__`` and returns a ``str``) is not supported.
Proposed variations
===
It was suggested to let ``%s`` accept numbers, but since numbers have their own
format codes this idea was discarded.
It has been suggested to use ``%b`` for bytes as well as ``%s``. This was
rejected as not adding any value either in clarity or simplicity.
It has been proposed to automatically use ``.encode('ascii','strict')`` for
``str`` arguments to ``%s``.
- Rejected as this would lead to intermittent failures. Better to have the
operation always fail so the trouble-spot can be correctly fixed.
It has been proposed to have ``%s`` return the ascii-encoded repr when the
value is a ``str`` (b'%s' % 'abc' -- b'abc').
- Rejected as this would lead to hard to debug failures far from the problem
site. Better to
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
At this I think this PEP has become a little too vague and abstract, and I think we'd probably be better served by getting more concrete: Problem: Some of Python 2's modules which are fundamentally necessary for interop with the broader internet, and the security thereof, are missing really important features. Right now Python 2 has a policy of getting absolutely new features. Solution: We're going to ignore that policy for a couple of pretty important features to that end. Here's my proposed list of such featuers: * hmac * constant_time_compare * os * Persisant FD for os.urandom() * ssl * SNI * SSLContext * A giant suite of constants from OpenSSL * The functions for checking a hostname against a certificate * The functions for finding the platform's certificate store Alex ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SSLSocket.send() for non-blocking sockets
On Wed, Mar 26, 2014 at 11:54 AM, Nikolaus Rath nikol...@rath.org wrote: 2. Change the behavior immediately, potentially breaking some applications that worked around it, but unbreaking others that relied on the documented behavior. If it's a functionality change that's likely to break code, would it be worth changing it only in 3.5, and documenting it as broken in 3.4 and earlier? ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] C code: %s vs %U
%s is a string. %U is unicode? If so, then %s should only be used when it is certain the string in question has no unicode in it? -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 3
I love it.
On Tue, Mar 25, 2014 at 6:37 PM, Ethan Furman et...@stoneleaf.us wrote:
Okay, I included that last round of comments (from late February).
Barring typos, this should be the final version.
Final comments?
-
PEP: 461
Title: Adding % formatting to bytes and bytearray
Version: $Revision$
Last-Modified: $Date$
Author: Ethan Furman et...@stoneleaf.us
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 2014-01-13
Python-Version: 3.5
Post-History: 2014-01-14, 2014-01-15, 2014-01-17, 2014-02-22, 2014-03-25
Resolution:
Abstract
This PEP proposes adding % formatting operations similar to Python 2's
``str``
type to ``bytes`` and ``bytearray`` [1]_ [2]_.
Rationale
=
While interpolation is usually thought of as a string operation, there are
cases where interpolation on ``bytes`` or ``bytearrays`` make sense, and the
work needed to make up for this missing functionality detracts from the
overall
readability of the code.
Motivation
==
With Python 3 and the split between ``str`` and ``bytes``, one small but
important area of programming became slightly more difficult, and much more
painful -- wire format protocols [3]_.
This area of programming is characterized by a mixture of binary data and
ASCII compatible segments of text (aka ASCII-encoded text). Bringing back a
restricted %-interpolation for ``bytes`` and ``bytearray`` will aid both in
writing new wire format code, and in porting Python 2 wire format code.
Common use-cases include ``dbf`` and ``pdf`` file formats, ``email``
formats, and ``FTP`` and ``HTTP`` communications, among many others.
Proposed semantics for ``bytes`` and ``bytearray`` formatting
=
%-interpolation
---
All the numeric formatting codes (such as ``%x``, ``%o``, ``%e``, ``%f``,
``%g``, etc.) will be supported, and will work as they do for str, including
the padding, justification and other related modifiers. The only difference
will be that the results from these codes will be ASCII-encoded text, not
unicode. In other words, for any numeric formatting code `%x`::
b%x % val
is equivalent to
(%x % val).encode(ascii)
Examples::
b'%4x' % 10
b' a'
b'%#4x' % 10
' 0xa'
b'%04X' % 10
'000A'
``%c`` will insert a single byte, either from an ``int`` in range(256), or
from
a ``bytes`` argument of length 1, not from a ``str``.
Examples::
b'%c' % 48
b'0'
b'%c' % b'a'
b'a'
``%s`` is included for two reasons: 1) `b` is already a format code for
``format`` numerics (binary), and 2) it will make 2/3 code easier as Python
2.x
code uses ``%s``; however, it is restricted in what it will accept::
- input type supports ``Py_buffer`` [6]_?
use it to collect the necessary bytes
- input type is something else?
use its ``__bytes__`` method [7]_ ; if there isn't one, raise a
``TypeError``
In particular, ``%s`` will not accept numbers (use a numeric format code for
that), nor ``str`` (encode it to ``bytes``).
Examples::
b'%s' % b'abc'
b'abc'
b'%s' % 'some string'.encode('utf8')
b'some string'
b'%s' % 3.14
Traceback (most recent call last):
...
TypeError: b'%s' does not accept numbers, use a numeric code instead
b'%s' % 'hello world!'
Traceback (most recent call last):
...
TypeError: b'%s' does not accept 'str', it must be encoded to `bytes`
``%a`` will call ``ascii()`` on the interpolated value. This is intended
as a debugging aid, rather than something that should be used in production.
Non-ASCII values will be encoded to either ``\xnn`` or ``\u``
representation. Use cases include developing a new protocol and writing
landmarks into the stream; debugging data going into an existing protocol
to see if the problem is the protocol itself or bad data; a fall-back for a
serialization format; or even a rudimentary serialization format when
defining ``__bytes__`` would not be appropriate [8].
.. note::
If a ``str`` is passed into ``%a``, it will be surrounded by quotes.
Unsupported codes
-
``%r`` (which calls ``__repr__`` and returns a ``str``) is not supported.
Proposed variations
===
It was suggested to let ``%s`` accept numbers, but since numbers have their
own
format codes this idea was discarded.
It has been suggested to use ``%b`` for bytes as well as ``%s``. This was
rejected as not adding any value either in clarity or simplicity.
It has been proposed to automatically use ``.encode('ascii','strict')`` for
``str`` arguments to ``%s``.
- Rejected as this would lead to intermittent failures. Better to have
the
operation always fail so the trouble-spot can be correctly fixed.
It has been proposed to
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 3/25/2014 6:15 PM, Nick Coghlan wrote: I am not sure how this meme got started, but let me be clear: the proposed policy DOES NOT provide blanket permission to break backwards compatibility in the affected modules. It only allows ADDING new features to bring these modules into line with their Python 3 counterparts, making it easier for third party packages like requests to do the right thing in a cross-version compatible way. I think the problem is that 'no new features' covers two quite different types of 'new' lumped together: change-new and new-new. New-new adds something completely new and does not break any sensible behavior that we worry about. We freely add new-new features with each new version. We do not add them to maintenance releases because then maintenance releases would be new versions. There are also considerations of incubation time, and the increased possibility of regressions with candidate-release versus alpha-beta-candidate-release schedules. Change-new additions changing existing behavior (like changing a default) and can therefore break code that we care about. Even for new versions, we are wary them and may require deprecation. One of the ironies of change-new is that it is much more similar to change-fix, which is allowed as maintenance, than new-new is. The tracker controversies are about issues on the borderline between change-fix and change-new. The original version of the PEP reference 434 as precedent. While partly valid, the reference could have fed the confusion because 434 does not draw the distinction given above. Indeed, part of the point of the PEP is that separating change-fix from change-new is difficult. As far as regressions go, a big new-new patch touching several untested modules worries me more than a small change-??? patch tweaking something in only one module, especially after writing tests for the module. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SSLSocket.send() for non-blocking sockets
Chris Angelico ros...@gmail.com writes: On Wed, Mar 26, 2014 at 11:54 AM, Nikolaus Rath nikol...@rath.org wrote: 2. Change the behavior immediately, potentially breaking some applications that worked around it, but unbreaking others that relied on the documented behavior. If it's a functionality change that's likely to break code, would it be worth changing it only in 3.5, and documenting it as broken in 3.4 and earlier? Yes, that's what I meant. I don't think changing it in 3.4 is an option at all. Best, -Nikolaus -- Encrypted emails preferred. PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C »Time flies like an arrow, fruit flies like a Banana.« ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
