Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-08 Thread M.-A. Lemburg
On 01.09.2014 10:09, Nick Coghlan wrote: On 1 September 2014 17:13, Christian Heimes christ...@python.org wrote: On 01.09.2014 08:44, Nick Coghlan wrote: Yes, it would have exactly the same security failure modes as sitecustomize, except it would only fire if the application imported the ssl

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-08 Thread Gregory P. Smith
On Wed, Sep 3, 2014 at 3:48 PM, Stephen J. Turnbull step...@xemacs.org wrote: Guido van Rossum writes: lot: five years ago (when I worked at Google!) it was common to find internal services that required SSL but had a misconfigured certificate, and the only way to access those services

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-04 Thread Antoine Pitrou
On Thu, 4 Sep 2014 13:11:38 +1000 Nick Coghlan ncogh...@gmail.com wrote: That leaves Python 2.7, and I have to say I'm now persuaded that a backport (including any required httplib and urllib features) is the right way to go. One of the tasks I'd been dreading as a follow-on from PEP 466 was

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-04 Thread Nick Coghlan
On 4 September 2014 22:39, Antoine Pitrou solip...@pitrou.net wrote: On Thu, 4 Sep 2014 13:11:38 +1000 Nick Coghlan ncogh...@gmail.com wrote: That leaves Python 2.7, and I have to say I'm now persuaded that a backport (including any required httplib and urllib features) is the right way to

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Cory Benfield
On 3 September 2014 01:19, Antoine Pitrou solip...@pitrou.net wrote: Keeping saying it doesn't make it magically true. Sure, but it *is* true, at the very least for HTTP. RFC 2818 (HTTP over TLS) has the following language in section 3.1: If the hostname is available, the client MUST check it

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Nick Coghlan
On 3 Sep 2014 18:28, Cory Benfield c...@lukasa.co.uk wrote: This is definitely true, and this change is both. The only question that matters is whether we believe we're doing users a service by breaking their code. I'd argue, along with Glyph, Alex and Donald, that we are. I've been on the

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Antoine Pitrou
On Wed, 3 Sep 2014 20:34:32 +1000 Nick Coghlan ncogh...@gmail.com wrote: The backwards compatibility argument only applies to Python 2 maintenance releases (where dreid indicated an intention to request backporting the change), and there I'm quite happy to take the position of use requests,

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Antoine Pitrou
On Tue, 02 Sep 2014 21:29:16 -0400 R. David Murray rdmur...@bitdance.com wrote: The top proposal so far is an sslcustomize.py file that could be used to either decrease or increase the default security. This is a much less handy solution than application options (eg, curl, wget) that allow

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread R. David Murray
On Wed, 03 Sep 2014 16:31:13 +0200, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 02 Sep 2014 21:29:16 -0400 R. David Murray rdmur...@bitdance.com wrote: The top proposal so far is an sslcustomize.py file that could be used to either decrease or increase the default security. This is

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Ethan Furman
On 09/03/2014 08:58 AM, R. David Murray wrote: I'm OK with letting go of this invalid-cert issue myself, given the lack of negative feedback Twisted got. I'll just keep my fingers crossed. I apologize if I missed this point, but if we have the source code then it is possible to go in and

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Alex Gaynor
Ethan Furman ethan at stoneleaf.us writes: I apologize if I missed this point, but if we have the source code then it is possible to go in and directly modify the application/utility to be able to talk over https to a router with an invalid certificate? This is an option when creating the

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Ethan Furman
On 09/03/2014 10:15 AM, Alex Gaynor wrote: Ethan Furman writes: I apologize if I missed this point, but if we have the source code then it is possible to go in and directly modify the application/utility to be able to talk over https to a router with an invalid certificate? This is an option

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Guido van Rossum
On Wed, Sep 3, 2014 at 8:58 AM, R. David Murray rdmur...@bitdance.com wrote: I'm OK with letting go of this invalid-cert issue myself, given the lack of negative feedback Twisted got. I'll just keep my fingers crossed. I'm with this sentiment (cautiously +1) -- and not just because of

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Antoine Pitrou
On Wed, 3 Sep 2014 10:54:55 -0700 Guido van Rossum gu...@python.org wrote: Let's take the plunge on this issue for the next 2.7 release (3.5 being a done deal). I'm entirely against this. Yes, some people will find that they have an old script accessing an old service which breaks. Surely

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Donald Stufft
On Sep 3, 2014, at 1:54 PM, Guido van Rossum gu...@python.org wrote: On Wed, Sep 3, 2014 at 8:58 AM, R. David Murray rdmur...@bitdance.com mailto:rdmur...@bitdance.com wrote: I'm OK with letting go of this invalid-cert issue myself, given the lack of negative feedback Twisted got. I'll

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread R. David Murray
On Wed, 03 Sep 2014 20:37:38 +0200, Antoine Pitrou solip...@pitrou.net wrote: On Wed, 3 Sep 2014 10:54:55 -0700 Guido van Rossum gu...@python.org wrote: Today (working at Dropbox, a much smaller company!) I don't even remember the last time I had to deal with such a browser complaint --

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Christian Heimes
On 03.09.2014 19:29, Ethan Furman wrote: Excellent. Last question (I hope): it is possible to (easily) create an SSLContext that will verify against a self-signed certificate? Yes: context = ssl.create_default_context(cafile=/path/to/selfsigned.pem) That works iff the certificate is

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread R. David Murray
On Wed, 03 Sep 2014 10:09:36 -0700, Ethan Furman et...@stoneleaf.us wrote: On 09/03/2014 08:58 AM, R. David Murray wrote: I'm OK with letting go of this invalid-cert issue myself, given the lack of negative feedback Twisted got. I'll just keep my fingers crossed. I apologize if I missed

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Guido van Rossum
Antoine, I think we are well past the point where arguments can sway positions. There clearly is no agreement on this issue. So please treat my post as a BDFL tie-breaker. I will just give you one thing to ponder -- those small/non-profit websites that can't afford proper certs are exactly the

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Christian Heimes
On 03.09.2014 19:54, Guido van Rossum wrote: Let's take the plunge on this issue for the next 2.7 release (3.5 being a done deal). Yes, some people will find that they have an old script accessing an old service which breaks. Surely some of the other changes in the same 2.7 bugfix release will

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Victor Stinner
2014-09-03 21:26 GMT+02:00 Christian Heimes christ...@python.org: On 03.09.2014 19:54, Guido van Rossum wrote: I'm +1 for Python 3.5 but -1 for Python 2.7. The SSLContext backport will landed in Python 2.7.9 (to be released). No Python 2 user is familiar with the feature yet. But more

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Guido van Rossum
OK, that changes my position for 2.7 (but not for 3.5). I had assumed there was a way to disable the cert check by changing one parameter to the urlopen() call. (And I had wanted to add that there should be a clear FAQ about the subject.) If this isn't possible that changes the situation. (But I

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Ethan Furman
On 09/03/2014 12:10 PM, R. David Murray wrote: On Wed, 03 Sep 2014 10:09:36 -0700, Ethan Furman et...@stoneleaf.us wrote: On 09/03/2014 08:58 AM, R. David Murray wrote: I'm OK with letting go of this invalid-cert issue myself, given the lack of negative feedback Twisted got. I'll just keep

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Christian Heimes
On 03.09.2014 21:37, Victor Stinner wrote: Thanks, you replied before I asked the question :-) (If certificates are validated by default, how do you disable the checks?) Sorry, I didn't follow the whole discussion and Python 2.7 changes related to security. Does Python 2.7 support loading

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Christian Heimes
On 03.09.2014 21:37, Guido van Rossum wrote: OK, that changes my position for 2.7 (but not for 3.5). I had assumed there was a way to disable the cert check by changing one parameter to the urlopen() call. (And I had wanted to add that there should be a clear FAQ about the subject.) If this

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Alex Gaynor
Guido van Rossum guido at python.org writes: OK, that changes my position for 2.7 (but not for 3.5). I had assumed there was a way to disable the cert check by changing one parameter to the urlopen() call. (And I had wanted to add that there should be a clear FAQ about the subject.) If this

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Stephen J. Turnbull
Guido van Rossum writes: lot: five years ago (when I worked at Google!) it was common to find internal services that required SSL but had a misconfigured certificate, and the only way to access those services was to override the browser complaints. Today (working at Dropbox, a much

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Benjamin Peterson
On Wed, Sep 3, 2014, at 13:37, Alex Gaynor wrote: Guido van Rossum guido at python.org writes: OK, that changes my position for 2.7 (but not for 3.5). I had assumed there was a way to disable the cert check by changing one parameter to the urlopen() call. (And I had wanted to add that

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Nick Coghlan
On 4 Sep 2014 04:39, Antoine Pitrou solip...@pitrou.net wrote: On Wed, 3 Sep 2014 10:54:55 -0700 Guido van Rossum gu...@python.org wrote: Let's take the plunge on this issue for the next 2.7 release (3.5 being a done deal). I'm entirely against this. Yes, some people will find that

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Nick Coghlan
On 4 Sep 2014 06:39, Alex Gaynor alex.gay...@gmail.com wrote: Guido van Rossum guido at python.org writes: OK, that changes my position for 2.7 (but not for 3.5). I had assumed there was a way to disable the cert check by changing one parameter to the urlopen() call. (And I had wanted to

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Antoine Pitrou
On Thu, 4 Sep 2014 09:19:56 +1000 Nick Coghlan ncogh...@gmail.com wrote: Python is routinely updated to bugfix releases by Linux distributions and other distribution channels, you usually have no say over what's shipped in those updates. This is not like changing the major version used

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Ethan Furman
On 09/03/2014 04:36 PM, Antoine Pitrou wrote: On Thu, 4 Sep 2014 09:19:56 +1000 Nick Coghlan ncogh...@gmail.com wrote: Python is routinely updated to bugfix releases by Linux distributions and other distribution channels, you usually have no say over what's shipped in those updates. This is

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Ethan Furman
On 09/03/2014 05:00 PM, Ethan Furman wrote: On 09/03/2014 04:36 PM, Antoine Pitrou wrote: On Thu, 4 Sep 2014 09:19:56 +1000 Nick Coghlan ncogh...@gmail.com wrote: Python is routinely updated to bugfix releases by Linux distributions and other distribution channels, you usually have no say

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-03 Thread Nick Coghlan
On 4 September 2014 10:00, Ethan Furman et...@stoneleaf.us wrote: On 09/03/2014 04:36 PM, Antoine Pitrou wrote: On Thu, 4 Sep 2014 09:19:56 +1000 Nick Coghlan ncogh...@gmail.com wrote: Python is routinely updated to bugfix releases by Linux distributions and other distribution channels,

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread Glyph Lefkowitz
On Aug 29, 2014, at 7:44 PM, Alex Gaynor alex.gay...@gmail.com wrote: Disabling verification entirely externally to the program, through a CLI flag or environment variable. I'm pretty down on this idea, the problem you hit is that it's a pretty blunt instrument to swing, and it's almost

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread Antoine Pitrou
On Tue, 2 Sep 2014 14:00:02 -0700 Glyph Lefkowitz gl...@twistedmatrix.com wrote: I would strongly recommend against such a mechanism. For what it's worth, Twisted simply unconditionally started verifying certificates in 14.0 with no disable switch, and (to my knowledge) literally no

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread Nick Coghlan
On 1 Sep 2014 16:05, Nick Coghlan ncogh...@gmail.com wrote: The final change would be to seed the context factory map appropriately for the standard library modules where we wanted to keep the *old* default: for modname in (nntplib, poplib, imaplib, ftplib, smtplib,

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread Alex Gaynor
Antoine Pitrou solipsis at pitrou.net writes: And how many people are using Twisted as an HTTPS client? (compared to e.g. Python's httplib, and all the third-party libraries building on it?) I don't think anyone could give an honest estimate of these counts, however there's two factors to

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread Antoine Pitrou
On Tue, 2 Sep 2014 22:16:18 + (UTC) Alex Gaynor alex.gay...@gmail.com wrote: Furthermore, disable verification is a nonsensical thing to do with TLS. It's not. For example, if you have an expired cert, all you can do AFAIK is to disable verification. It really is a nonsensical

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread Christian Heimes
On 02.09.2014 23:32, Antoine Pitrou wrote: Furthermore, disable verification is a nonsensical thing to do with TLS. It's not. For example, if you have an expired cert, all you can do AFAIK is to disable verification. It's possible to ignore or just warn about expired certs with simple verify

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread Nick Coghlan
On 3 Sep 2014 08:18, Alex Gaynor alex.gay...@gmail.com wrote: Antoine Pitrou solipsis at pitrou.net writes: And how many people are using Twisted as an HTTPS client? (compared to e.g. Python's httplib, and all the third-party libraries building on it?) I don't think anyone could

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread David Reid
Nick Coghlan ncoghlan at gmail.com writes: Creating *new* incompatibilities between Python 2 Python 3 is a major point of concern. Clearly this change should be backported to Python2. -David ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread R. David Murray
On Tue, 02 Sep 2014 22:16:18 -, Alex Gaynor alex.gay...@gmail.com wrote: This whole scenario seems to be predicated on a siutation where: You have a peer whose certificate you can't change, and you have a piece of code you can't change, and you're going to upgrade your Python

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread Glyph Lefkowitz
On Sep 2, 2014, at 4:01 PM, Nick Coghlan ncogh...@gmail.com wrote: On 3 Sep 2014 08:18, Alex Gaynor alex.gay...@gmail.com wrote: Antoine Pitrou solipsis at pitrou.net writes: And how many people are using Twisted as an HTTPS client? (compared to e.g. Python's httplib, and all

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread Nick Coghlan
On 3 Sep 2014 09:08, David Reid dr...@dreid.org wrote: Nick Coghlan ncoghlan at gmail.com writes: Creating *new* incompatibilities between Python 2 Python 3 is a major point of concern. Clearly this change should be backported to Python2. Proposing to break backwards compatibility in a

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread Glyph Lefkowitz
On Sep 2, 2014, at 4:28 PM, Nick Coghlan ncogh...@gmail.com wrote: On 3 Sep 2014 09:08, David Reid dr...@dreid.org wrote: Nick Coghlan ncoghlan at gmail.com writes: Creating *new* incompatibilities between Python 2 Python 3 is a major point of concern. Clearly this change

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread Donald Stufft
On Sep 2, 2014, at 7:47 PM, Glyph Lefkowitz gl...@twistedmatrix.com wrote: On Sep 2, 2014, at 4:28 PM, Nick Coghlan ncogh...@gmail.com mailto:ncogh...@gmail.com wrote: On 3 Sep 2014 09:08, David Reid dr...@dreid.org mailto:dr...@dreid.org wrote: Nick Coghlan ncoghlan at gmail.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread Antoine Pitrou
On Tue, 2 Sep 2014 16:47:35 -0700 Glyph Lefkowitz gl...@twistedmatrix.com wrote: On Sep 2, 2014, at 4:28 PM, Nick Coghlan ncogh...@gmail.com wrote: On 3 Sep 2014 09:08, David Reid dr...@dreid.org wrote: Nick Coghlan ncoghlan at gmail.com writes: Creating *new* incompatibilities

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread Terry Reedy
On 9/2/2014 7:47 PM, Glyph Lefkowitz wrote: On Sep 2, 2014, at 4:28 PM, Nick Coghlan ncogh...@gmail.com mailto:ncogh...@gmail.com wrote: On 3 Sep 2014 09:08, David Reid dr...@dreid.org mailto:dr...@dreid.org wrote: Clearly this change should be backported to Python2. Proposing to break

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread R. David Murray
On Tue, 02 Sep 2014 20:59:54 -0400, Terry Reedy tjre...@udel.edu wrote: On 9/2/2014 7:47 PM, Glyph Lefkowitz wrote: On Sep 2, 2014, at 4:28 PM, Nick Coghlan ncogh...@gmail.com mailto:ncogh...@gmail.com wrote: On 3 Sep 2014 09:08, David Reid dr...@dreid.org mailto:dr...@dreid.org

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-02 Thread Stephen J. Turnbull
Antoine Pitrou writes: On Tue, 2 Sep 2014 16:47:35 -0700 Glyph Lefkowitz gl...@twistedmatrix.com wrote: As we keep saying, this is not a break in backwards compatibility, it's a bug fix. Keeping saying it doesn't make it magically true. It's not magically true, it is just true.

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Nick Coghlan
On 1 September 2014 11:10, R. David Murray rdmur...@bitdance.com wrote: It sounds like this would address my concerns as well (I don't really care *how* it is implemented as long as I don't have to touch the code of a third party application when I upgrade my python version to 3.5...remember,

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Paul Moore
On 31 August 2014 23:10, Nick Coghlan ncogh...@gmail.com wrote: Assuming sslcustomize was in site-packages rather than the standard library directories, you would also be able to use virtual environments with an appropriate sslcustomize module to disable cert checking even if the application

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Nick Coghlan
On 1 September 2014 16:07, Paul Moore p.f.mo...@gmail.com wrote: On 31 August 2014 23:10, Nick Coghlan ncogh...@gmail.com wrote: Assuming sslcustomize was in site-packages rather than the standard library directories, you would also be able to use virtual environments with an appropriate

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Christian Heimes
On 01.09.2014 08:44, Nick Coghlan wrote: Yes, it would have exactly the same security failure modes as sitecustomize, except it would only fire if the application imported the ssl module. The -S and -I switches would need to disable the implied sslcustomize, just as they disable import

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Nick Coghlan
On 1 September 2014 17:13, Christian Heimes christ...@python.org wrote: On 01.09.2014 08:44, Nick Coghlan wrote: Yes, it would have exactly the same security failure modes as sitecustomize, except it would only fire if the application imported the ssl module. The -S and -I switches would

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Antoine Pitrou
Le 01/09/2014 10:09, Nick Coghlan a écrit : On 1 September 2014 17:13, Christian Heimes christ...@python.org wrote: On 01.09.2014 08:44, Nick Coghlan wrote: Yes, it would have exactly the same security failure modes as sitecustomize, except it would only fire if the application imported the

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Chris Angelico
On Mon, Sep 1, 2014 at 10:41 PM, Antoine Pitrou anto...@python.org wrote: Not sure why. Just put another module named ssl in sys.modules directly. You can also monkeypatch the genuine ssl module. That has to be done inside the same process. But imagine this scenario: You have a program that

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Antoine Pitrou
On Mon, 1 Sep 2014 23:24:39 +1000 Chris Angelico ros...@gmail.com wrote: On Mon, Sep 1, 2014 at 10:41 PM, Antoine Pitrou anto...@python.org wrote: Not sure why. Just put another module named ssl in sys.modules directly. You can also monkeypatch the genuine ssl module. That has to be done

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Chris Angelico
On Mon, Sep 1, 2014 at 11:34 PM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 1 Sep 2014 23:24:39 +1000 Chris Angelico ros...@gmail.com wrote: On Mon, Sep 1, 2014 at 10:41 PM, Antoine Pitrou anto...@python.org wrote: Not sure why. Just put another module named ssl in sys.modules

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Antoine Pitrou
On Mon, 1 Sep 2014 23:42:10 +1000 Chris Angelico ros...@gmail.com wrote: On Mon, Sep 1, 2014 at 11:34 PM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 1 Sep 2014 23:24:39 +1000 Chris Angelico ros...@gmail.com wrote: On Mon, Sep 1, 2014 at 10:41 PM, Antoine Pitrou anto...@python.org

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Nick Coghlan
On 2 Sep 2014 00:08, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 1 Sep 2014 23:42:10 +1000 Chris Angelico ros...@gmail.com wrote: That has to be done inside the same process. But imagine this scenario: You have a program that gets invoked as root (or some other user than

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Antoine Pitrou
On Tue, 2 Sep 2014 00:53:11 +1000 Nick Coghlan ncogh...@gmail.com wrote: On 2 Sep 2014 00:08, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 1 Sep 2014 23:42:10 +1000 Chris Angelico ros...@gmail.com wrote: That has to be done inside the same process. But imagine this

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Nick Coghlan
On 2 Sep 2014 00:59, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 2 Sep 2014 00:53:11 +1000 Nick Coghlan ncogh...@gmail.com wrote: To be frank I don't understand what you're arguing about. When I said shadowing ssl can be tricky to arrange, Chris correctly interpreted it as

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Donald Stufft
On Sep 1, 2014, at 11:35 AM, Nick Coghlan ncogh...@gmail.com wrote: On 2 Sep 2014 00:59, Antoine Pitrou solip...@pitrou.net mailto:solip...@pitrou.net wrote: On Tue, 2 Sep 2014 00:53:11 +1000 Nick Coghlan ncogh...@gmail.com mailto:ncogh...@gmail.com wrote: To be frank I

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Christian Heimes
On 01.09.2014 17:35, Nick Coghlan wrote: Oh, now I get what you mean - yes, sitecustomize already poses the same kind of problem as the proposed sslcustomize (hence the existence of the related command line options). If an attacker is able to place a module like sitecustomize.py in an import

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Donald Stufft
On Sep 1, 2014, at 1:01 PM, Christian Heimes christ...@python.org wrote: On 01.09.2014 17:35, Nick Coghlan wrote: Oh, now I get what you mean - yes, sitecustomize already poses the same kind of problem as the proposed sslcustomize (hence the existence of the related command line options).

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-09-01 Thread Nick Coghlan
On 2 Sep 2014 03:08, Donald Stufft don...@stufft.io wrote: On Sep 1, 2014, at 1:01 PM, Christian Heimes christ...@python.org wrote: On 01.09.2014 17:35, Nick Coghlan wrote: Oh, now I get what you mean - yes, sitecustomize already poses the same kind of problem as the proposed sslcustomize

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Nick Coghlan
On 31 August 2014 12:21, R. David Murray rdmur...@bitdance.com wrote: On Sun, 31 Aug 2014 03:25:25 +0200, Antoine Pitrou solip...@pitrou.net wrote: On Sun, 31 Aug 2014 09:26:30 +1000 Nick Coghlan ncogh...@gmail.com wrote: In relation to changing the Python CLI API to offer some of the

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Donald Stufft
On Aug 31, 2014, at 2:09 AM, Nick Coghlan ncogh...@gmail.com wrote: At the same time, we need to account for the fact that most existing organisations still trust in perimeter defence for their internal network security, and hence tolerate (or even actively encourage) the use of unsecured

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Nick Coghlan
On 31 August 2014 12:21, R. David Murray rdmur...@bitdance.com wrote: Do those knobs allow one to instruct urllib to accept an invalid certificate without changing the program code? My first reply ended up being a context dump of the challenges created by legacy corporate intranets that may not

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Nick Coghlan
On 31 August 2014 16:16, Donald Stufft don...@stufft.io wrote: On Aug 31, 2014, at 2:09 AM, Nick Coghlan ncogh...@gmail.com wrote: At the same time, we need to account for the fact that most existing organisations still trust in perimeter defence for their internal network security, and

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Cory Benfield
On 31 August 2014 07:45, Nick Coghlan ncogh...@gmail.com wrote: There's also the fact that most corporate Python users are unlikely to know that PyPI exists, let alone that it contains a module called requests that does SSL certificate validation by default. Those of us in the corporate world

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Christian Heimes
On 30.08.2014 00:22, Antoine Pitrou wrote: SSL_CERT_DIR and SSL_CERT_FILE are used, if set, when SSLContext.load_verify_locations() is called. Actually, come to think of it, this allows us to write a better test for that method. Patch welcome! The environment vars are used only when

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread R. David Murray
On Sun, 31 Aug 2014 16:45:42 +1000, Nick Coghlan ncogh...@gmail.com wrote: On 31 August 2014 16:16, Donald Stufft don...@stufft.io wrote: On Aug 31, 2014, at 2:09 AM, Nick Coghlan ncogh...@gmail.com wrote: At the same time, we need to account for the fact that most existing

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Christian Heimes
On 31.08.2014 16:16, R. David Murray wrote: Self -signed certificates are not crazy in an internal corporate environment even when properly playing the defense in depth game. Once you've acked the cert the first time, you will be warned if it changes (like an ssh host key). Sure, as Nick

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Paul Moore
On 31 August 2014 17:27, Christian Heimes christ...@python.org wrote: It's very simple to trust a self-signed certificate: just download it and stuff it into the trust store. Stuff it into the trust store is the hard bit, though. I have honestly no idea how to do that. Or if it's temporary

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Christian Heimes
On 31.08.2014 08:24, Nick Coghlan wrote: To answer David's specific question, the existing knobs at the OpenSSL level (SSL_CERT_DIR and SSL_CERT_FILE ) let people add an internal CA, opt out of the default CA system, and trust *specific* self-signed certs. This works only on Unix platforms

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Antoine Pitrou
Le 31/08/2014 19:03, Paul Moore a écrit : On 31 August 2014 17:27, Christian Heimes christ...@python.org wrote: It's very simple to trust a self-signed certificate: just download it and stuff it into the trust store. Stuff it into the trust store is the hard bit, though. I have honestly no

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Paul Moore
On 31 August 2014 18:29, Antoine Pitrou anto...@python.org wrote: If an application has special needs that require trusting a self-signed certificate, then it should expose a configuration setting to let users specify the cert's location. I can't see how that would be something the application

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Antoine Pitrou
Le 31/08/2014 20:28, Paul Moore a écrit : I can't see how that would be something the application would know. For example, pip allows me to specify an alternate cert bundle but not a single additional cert. So IIUC, I can't use my local index that serves https using a self-signed cert. I'd find

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Paul Moore
On 31 August 2014 19:37, Antoine Pitrou anto...@python.org wrote: Well, it's certainly pip's responsibility more than Python's. What would Python do? Provide a setting that would blindly add a cert for all uses of httplib? That's more or less my point, pip doesn't have that much better idea

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Antoine Pitrou
Le 31/08/2014 21:12, Paul Moore a écrit : On 31 August 2014 19:37, Antoine Pitrou anto...@python.org wrote: Well, it's certainly pip's responsibility more than Python's. What would Python do? Provide a setting that would blindly add a cert for all uses of httplib? That's more or less my

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Christian Heimes
On 31.08.2014 19:29, Antoine Pitrou wrote: You certainly shouldn't do so. If an application has special needs that require trusting a self-signed certificate, then it should expose a configuration setting to let users specify the cert's location. Stuffing self-signed certs into the system

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Paul Moore
On 31 August 2014 21:15, Antoine Pitrou anto...@python.org wrote: What do you call your local cert store? I was referring to Christian's comment It's very simple to trust a self-signed certificate: just download it and stuff it into the trust store. From his recent response, I guess he meant

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Nick Coghlan
On 1 Sep 2014 06:32, Paul Moore p.f.mo...@gmail.com wrote: On 31 August 2014 21:15, Antoine Pitrou anto...@python.org wrote: What do you call your local cert store? I was referring to Christian's comment It's very simple to trust a self-signed certificate: just download it and stuff it

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Christian Heimes
On 31.08.2014 08:09, Nick Coghlan wrote: As Antoine says here, I'm also opposed to adding more Python specific configuration options. However, I think there may be something worthwhile we can do that's closer to the way browsers work, and has the significant benefit of being implementable as a

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Antoine Pitrou
Le 31/08/2014 23:41, Nick Coghlan a écrit : Right, this is why I came to the conclusion we need to follow the browser vendors lead here and support a per-user Python specific supplementary certificate cache before we can start validating certs by default at the *Python* level. There are still

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Christian Heimes
On 31.08.2014 22:30, Paul Moore wrote: On 31 August 2014 21:15, Antoine Pitrou anto...@python.org wrote: What do you call your local cert store? I was referring to Christian's comment It's very simple to trust a self-signed certificate: just download it and stuff it into the trust store.

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Nick Coghlan
On 1 Sep 2014 07:43, Christian Heimes christ...@python.org wrote: On 31.08.2014 08:09, Nick Coghlan wrote: As Antoine says here, I'm also opposed to adding more Python specific configuration options. However, I think there may be something worthwhile we can do that's closer to the way

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Donald Stufft
On Aug 31, 2014, at 5:43 PM, Christian Heimes christ...@python.org wrote: On 31.08.2014 08:09, Nick Coghlan wrote: As Antoine says here, I'm also opposed to adding more Python specific configuration options. However, I think there may be something worthwhile we can do that's closer to the

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Nick Coghlan
On 1 Sep 2014 08:15, Donald Stufft don...@stufft.io wrote: On Aug 31, 2014, at 5:43 PM, Christian Heimes christ...@python.org wrote: Companies could install them in a system global directory on their servers. Users could put them in their own user site directory and even each virtual env

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread R. David Murray
On Mon, 01 Sep 2014 08:10:58 +1000, Nick Coghlan ncogh...@gmail.com wrote: On 1 Sep 2014 07:43, Christian Heimes christ...@python.org wrote: On 31.08.2014 08:09, Nick Coghlan wrote: As Antoine says here, I'm also opposed to adding more Python specific configuration options. However, I

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-30 Thread M.-A. Lemburg
On 30.08.2014 04:44, Alex Gaynor wrote: Thanks for the rapid feedback everyone! I want to summarize the action items and discussion points that have come up so far: To add to the PEP: * Emit a warning in 3.4.next for cases that would raise a Exception in 3.5 * Clearly state that the

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-30 Thread Antoine Pitrou
On Sat, 30 Aug 2014 12:19:11 +0200 M.-A. Lemburg m...@egenix.com wrote: To add to the PEP: * Emit a warning in 3.4.next for cases that would raise a Exception in 3.5 * Clearly state that the existing OpenSSL environment variables will be respected for setting the trust root I'd

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-30 Thread M.-A. Lemburg
On 30.08.2014 12:40, Antoine Pitrou wrote: On Sat, 30 Aug 2014 12:19:11 +0200 M.-A. Lemburg m...@egenix.com wrote: To add to the PEP: * Emit a warning in 3.4.next for cases that would raise a Exception in 3.5 * Clearly state that the existing OpenSSL environment variables will be

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-30 Thread Paul Moore
30 August 2014 03:44, Alex Gaynor alex.gay...@gmail.com wrote: Discussion points: * Disabling verification entirely externally to the program, through a CLI flag or environment variable. I'm pretty down on this idea, the problem you hit is that it's a pretty blunt instrument to

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-30 Thread Antoine Pitrou
On Sat, 30 Aug 2014 12:46:47 +0200 M.-A. Lemburg m...@egenix.com wrote: The change is to the OpenSSL API, not the OpenSSL lib. By setting the variable you enable a few special calls to the config loader functions in OpenSSL when calling the initializer it:

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-30 Thread M.-A. Lemburg
On 30.08.2014 12:55, Antoine Pitrou wrote: On Sat, 30 Aug 2014 12:46:47 +0200 M.-A. Lemburg m...@egenix.com wrote: The change is to the OpenSSL API, not the OpenSSL lib. By setting the variable you enable a few special calls to the config loader functions in OpenSSL when calling the

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-30 Thread R. David Murray
On Sat, 30 Aug 2014 14:03:57 +0200, M.-A. Lemburg m...@egenix.com wrote: On 30.08.2014 12:55, Antoine Pitrou wrote: On Sat, 30 Aug 2014 12:46:47 +0200 M.-A. Lemburg m...@egenix.com wrote: That use case should be served with the SSL_CERT_DIR and SSL_CERT_FILE env vars (or, better, by

  1   2   >