Re: [Python-Dev] PEP process entry point and ill fated initiatives
On 30/11/2013 03:39, Nick Coghlan wrote: On 30 November 2013 01:25, Kristján Valur Jónsson wrote: I know that Anatoly himself is a subject of long history here, but I myself have felt lessening affinity to the dev community in recent years. It feels like it is increasingly shutting itself in. Are you sure it isn't just that the focus of development has shifted to matters that aren't of interest or relevance to you? Many (perhaps even most) problems in Python don't require changes at the language or standard library level. We have cycle times measured in months, and impact times measured in years (especially since core development switched to Python 3 only mode for feature development). That's not typically something that is useful in day-to-day development tasks - it's only ever relevant in strategic terms. One thing that has changed for me personally, is that I've become far more blunt about refusing to respect those that explicitly (and vocally) refuse to respect us, yet seem to want to participate in core development anyway, and that's directly caused by Anatoly. He's still the only person who has been proposed for a permanent ban from all python.org controlled communication channels. That was averted after he voluntarily stopped annoying people for a while, but now he's back and I think the matter needs to be reconsidered. He refuses to sign the CLA that would allow him to contribute directly, yet still wants people to fix things for him. He refuses to read design documentation, yet still wants people to listen to his ideas. He refuses to care about other people's use cases, yet still wants people to care about his. As a case in point, Anatoly recently suggested that more diagrams in the documentation would be a good thing (http://bugs.python.org/issue19608). That's not an objectively bad idea, but producing and maintaining good diagrams is a high overhead activity, so we generally don't bother. When I suggested drawing some and sending a patch (I had forgotten about the CLA problem), Anatoly's response was that he's not a designer. So I countered with a suggestion that he explore what would be involved in adding the seqdiag and blockdiag sphinx extensions to our docs build process, since having those available would drastically lower the barrier to including and maintaining reasonable diagrams in the documentation, increasing the chance of such diagrams being included in the future. Silence. "Hey some diagrams would be helpful!" is not a useful contribution, it's stating the bleeding obvious. Even nominating some *specific* parts of the guide where a diagram would have helped Anatoly personally would have been useful. The technical change I suggested about figuring out what we'd need to change to enable those extensions would *definitely* have been useful. Another couple of incidents recently occurred on distutils-sig, where Anatoly started second guessing the decision to work on PyPI 2 as a test-driven-development-from-day-one incrementally developed and released system, rather than trying to update the existing fragile PyPI code base directly, as well as complaining about the not-accessible-to-end-users design docs for the proposed end-to-end security model for PyPI. It would be one thing if he was voicing those concerns on his own blog (it's a free internet, he can do what he likes anywhere else). It's a problem when he's doing it on distutils-sig and the project issue trackers. This isn't a matter of a naive newcomer that doesn't know any better. This is someone who has had PSF board members sit down with them at PyCon US to explain the CLA and why it is the way it is, who has had core developers offer them direct advice on how to propose suggestions in a way that is more likely to get people to listen, and when major issues have occurred in the past, we've even gone hunting for people to talk to him in his native language to make sure it wasn't a language barrier that was the root cause of the problem. *None* of it has resulted in any signficant improvement in his behaviour. Contributor time and emotional energy are the most precious resources an open source project has, and Anatoly is recklessly wasteful of both. We've spent years trying to coach him on being an effective collaborator and contributor, and it hasn't worked. This isn't a democracy, and neither is it a place for arbitrary people to get therapy on their inability to have any empathy for another person's point of view - in the end, passion for the language isn't enough, people have to demonstrate an ability to learn and be respectful of other people's time and energy, and Anatoly has well and truly proven he doesn't have either of those. Anatoly has the entire rest of the internet to play in, we shouldn't have to put up with his disruptions when we're actually trying to get stuff done. Regards, Nick. FWIW I agree entirely with your sentiments. Several times recently I've seen him in action on the bug tracker. I
Re: [Python-Dev] PEP process entry point and ill fated initiatives
Thanks for this long explanation, Nick. For someone that is not a compulsive reader of python-dev it certainly helps by putting things in perspective. I think the problem you describe is a singular one that needs to be dealt with using singular methods. My own personal complaints, have other causes, I hope, and I see now that bringing the two up as being somehow related is both incorrect and unwise. I'm sorry for stirring things up, I'll try to show more restraint in the future :) Cheers, Kristján -Original Message- From: Nick Coghlan [mailto:ncogh...@gmail.com] Sent: 30. nóvember 2013 03:39 To: Kristján Valur Jónsson Cc: Antoine Pitrou; python-dev@python.org Subject: Re: [Python-Dev] PEP process entry point and ill fated initiatives On 30 November 2013 01:25, Kristján Valur Jónsson wrote: > I know that Anatoly himself is a subject of long history here, but I > myself have felt lessening affinity to the dev community in recent > years. It feels like it is increasingly shutting itself in. Are you sure it isn't just that the focus of development has shifted to matters that aren't of interest or relevance to you? Many (perhaps even most) problems in Python don't require changes at the language or standard library level. We have cycle times measured in months, and impact times measured in years (especially since core development switched to Python 3 only mode for feature development). That's not typically something that is useful in day-to-day development tasks - it's only ever relevant in strategic terms. One thing that has changed for me personally, is that I've become far more blunt about refusing to respect those that explicitly (and vocally) refuse to respect us, yet seem to want to participate in core development anyway, and that's directly caused by Anatoly. He's still the only person who has been proposed for a permanent ban from all python.org controlled communication channels. That was averted after he voluntarily stopped annoying people for a while, but now he's back and I think the matter needs to be reconsidered. He refuses to sign the CLA that would allow him to contribute directly, yet still wants people to fix things for him. He refuses to read design documentation, yet still wants people to listen to his ideas. He refuses to care about other people's use cases, yet still wants people to care about his. As a case in point, Anatoly recently suggested that more diagrams in the documentation would be a good thing (http://bugs.python.org/issue19608). That's not an objectively bad idea, but producing and maintaining good diagrams is a high overhead activity, so we generally don't bother. When I suggested drawing some and sending a patch (I had forgotten about the CLA problem), Anatoly's response was that he's not a designer. So I countered with a suggestion that he explore what would be involved in adding the seqdiag and blockdiag sphinx extensions to our docs build process, since having those available would drastically lower the barrier to including and maintaining reasonable diagrams in the documentation, increasing the chance of such diagrams being included in the future. Silence. "Hey some diagrams would be helpful!" is not a useful contribution, it's stating the bleeding obvious. Even nominating some *specific* parts of the guide where a diagram would have helped Anatoly personally would have been useful. The technical change I suggested about figuring out what we'd need to change to enable those extensions would *definitely* have been useful. Another couple of incidents recently occurred on distutils-sig, where Anatoly started second guessing the decision to work on PyPI 2 as a test-driven-development-from-day-one incrementally developed and released system, rather than trying to update the existing fragile PyPI code base directly, as well as complaining about the not-accessible-to-end-users design docs for the proposed end-to-end security model for PyPI. It would be one thing if he was voicing those concerns on his own blog (it's a free internet, he can do what he likes anywhere else). It's a problem when he's doing it on distutils-sig and the project issue trackers. This isn't a matter of a naive newcomer that doesn't know any better. This is someone who has had PSF board members sit down with them at PyCon US to explain the CLA and why it is the way it is, who has had core developers offer them direct advice on how to propose suggestions in a way that is more likely to get people to listen, and when major issues have occurred in the past, we've even gone hunting for people to talk to him in his native language to make sure it wasn't a language barrier that was the root cause of the problem. *None* of it has resulted in any signficant improvement in his behaviour. Contributor time and emotional energy are the most precious resources an open source project has, and Anatoly is recklessly wasteful of both. We've spent y
[Python-Dev] Verification of SSL cert and hostname made easy
Hi,
Larry has granted me a special pardon to add an outstanding fix for SSL,
http://bugs.python.org/issue19509 . Right now most stdlib modules
(ftplib, imaplib, nntplib, poplib, smtplib) neither support server name
indication (SNI) nor check the subject name of the peer's certificate
properly. The second issue is a major loop-hole because it allows
man-in-the-middle attack despite CERT_REQUIRED.
With CERT_REQUIRED OpenSSL verifies that the peer's certificate is
directly or indirectly signed by a trusted root certification authority.
With Python 3.4 the ssl module is able to use/load the system's trusted
root certs on all major systems (Linux, Mac, BSD, Windows). On Linux and
BSD it requires a properly configured system openssl to locate the root
certs. This usually works out of the box. On Mac Apple's openssl build
is able to use the keychain API of OSX. I have added code for Windows'
system store.
SSL socket code usually looks like this:
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
context.verify_mode = ssl.CERT_REQUIRED
# new, by default it loads certs trusted for Purpose.SERVER_AUTH
context.load_default_certs()
sock = socket.create_connection(("example.net", 443))
sslsock = context.wrap_socket(sock)
SSLContext.wrap_socket() wraps an ordinary socket into a SSLSocket. With
verify_mode = CERT_REQUIRED OpenSSL ensures that the peer's SSL
certificate is signed by a trusted root CA. In this example one very
important step is missing. The peer may return *ANY* signed certificate
for *ANY* hostname. These lines do NOT check that the certificate's
information match "example.net". An attacker can use any arbitrary
certificate (e.g. for "www.evil.net"), get it signed and abuse it for
MitM attacks on "mail.python.org".
http://docs.python.org/3/library/ssl.html#ssl.match_hostname must be
used to verify the cert. It's easy to forget it...
I have thought about multiple ways to fix the issue. At first I added a
new argument "check_hostname" to all affected modules and implemented
the check manually. For every module I had to modify several places for
SSL and STARTTLS and add / change about 10 lines. The extra lines are
required to properly shutdown and close the connection when the cert
doesn't match the hostname. I don't like the solution because it's
tedious. Every 3rd party author has to copy the same code, too.
Then I came up with a better solution:
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
context.verify_mode = ssl.CERT_REQUIRED
context.load_default_certs()
context.check_hostname = True # <-- NEW
sock = socket.create_connection(("example.net", 443))
# server_hostname is already used for SNI
sslsock = context.wrap_socket(sock, server_hostname="example.net")
This fix requires only a new SSLContext attribute and a small
modification to SSLSocket.do_handshake():
if self.context.check_hostname:
try:
match_hostname(self.getpeercert(), self.server_hostname)
except Exception:
self.shutdown(_SHUT_RDWR)
self.close()
raise
Pros:
* match_hostname() is done in one central place
* the cert is matched as early as possible
* no extra arguments for APIs, a context object is enough
* library developers just have to add server_hostname to get SNI and
hostname checks at the same time
* users of libraries can configure cert verification and checking on the
same object
* missing checks will not pass silently
Cons:
* Doesn't work with OpenSSL < 0.9.8f (released 2007) because older
versions lack SNI support. The ssl module raises an exception for
server_hostname if SNI is not supported.
The default settings for all stdlib modules will still be verify_mode =
CERT_NONE and check_hostname = False for maximum backward compatibility.
Python 3.4 comes with a new function ssl.create_default_context() that
returns a new context with best practice settings and loaded root CA
certs. The settings are TLS 1.0, no weak and insecure ciphers (no MD5,
no RC4), no compression (CRIME attack), CERT_REQUIRED and check_hostname
= True (for client side only).
http://bugs.python.org/issue19509 has a working patch for ftplib.
Comments?
Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Verification of SSL cert and hostname made easy
On 1 Dec 2013 04:32, "Christian Heimes" wrote:
>
> Hi,
>
> Larry has granted me a special pardon to add an outstanding fix for SSL,
> http://bugs.python.org/issue19509 . Right now most stdlib modules
> (ftplib, imaplib, nntplib, poplib, smtplib) neither support server name
> indication (SNI) nor check the subject name of the peer's certificate
> properly. The second issue is a major loop-hole because it allows
> man-in-the-middle attack despite CERT_REQUIRED.
>
> With CERT_REQUIRED OpenSSL verifies that the peer's certificate is
> directly or indirectly signed by a trusted root certification authority.
> With Python 3.4 the ssl module is able to use/load the system's trusted
> root certs on all major systems (Linux, Mac, BSD, Windows). On Linux and
> BSD it requires a properly configured system openssl to locate the root
> certs. This usually works out of the box. On Mac Apple's openssl build
> is able to use the keychain API of OSX. I have added code for Windows'
> system store.
>
> SSL socket code usually looks like this:
>
> context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
> context.verify_mode = ssl.CERT_REQUIRED
> # new, by default it loads certs trusted for Purpose.SERVER_AUTH
> context.load_default_certs()
>
> sock = socket.create_connection(("example.net", 443))
> sslsock = context.wrap_socket(sock)
>
> SSLContext.wrap_socket() wraps an ordinary socket into a SSLSocket. With
> verify_mode = CERT_REQUIRED OpenSSL ensures that the peer's SSL
> certificate is signed by a trusted root CA. In this example one very
> important step is missing. The peer may return *ANY* signed certificate
> for *ANY* hostname. These lines do NOT check that the certificate's
> information match "example.net". An attacker can use any arbitrary
> certificate (e.g. for "www.evil.net"), get it signed and abuse it for
> MitM attacks on "mail.python.org".
> http://docs.python.org/3/library/ssl.html#ssl.match_hostname must be
> used to verify the cert. It's easy to forget it...
>
>
> I have thought about multiple ways to fix the issue. At first I added a
> new argument "check_hostname" to all affected modules and implemented
> the check manually. For every module I had to modify several places for
> SSL and STARTTLS and add / change about 10 lines. The extra lines are
> required to properly shutdown and close the connection when the cert
> doesn't match the hostname. I don't like the solution because it's
> tedious. Every 3rd party author has to copy the same code, too.
>
> Then I came up with a better solution:
>
> context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
> context.verify_mode = ssl.CERT_REQUIRED
> context.load_default_certs()
> context.check_hostname = True # <-- NEW
>
> sock = socket.create_connection(("example.net", 443))
> # server_hostname is already used for SNI
> sslsock = context.wrap_socket(sock, server_hostname="example.net")
>
>
> This fix requires only a new SSLContext attribute and a small
> modification to SSLSocket.do_handshake():
>
> if self.context.check_hostname:
> try:
> match_hostname(self.getpeercert(), self.server_hostname)
> except Exception:
> self.shutdown(_SHUT_RDWR)
> self.close()
> raise
>
>
> Pros:
>
> * match_hostname() is done in one central place
> * the cert is matched as early as possible
> * no extra arguments for APIs, a context object is enough
> * library developers just have to add server_hostname to get SNI and
> hostname checks at the same time
> * users of libraries can configure cert verification and checking on the
> same object
> * missing checks will not pass silently
>
> Cons:
>
> * Doesn't work with OpenSSL < 0.9.8f (released 2007) because older
> versions lack SNI support. The ssl module raises an exception for
> server_hostname if SNI is not supported.
>
>
> The default settings for all stdlib modules will still be verify_mode =
> CERT_NONE and check_hostname = False for maximum backward compatibility.
> Python 3.4 comes with a new function ssl.create_default_context() that
> returns a new context with best practice settings and loaded root CA
> certs. The settings are TLS 1.0, no weak and insecure ciphers (no MD5,
> no RC4), no compression (CRIME attack), CERT_REQUIRED and check_hostname
> = True (for client side only).
>
> http://bugs.python.org/issue19509 has a working patch for ftplib.
>
> Comments?
If Larry is OK with it as RM (and it sounds like he is), +1 from me as well.
Cheers,
Nick.
>
> Christian
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Verification of SSL cert and hostname made easy
Sounds good.
Is another change for asyncio needed?
On Sat, Nov 30, 2013 at 1:54 PM, Nick Coghlan wrote:
>
> On 1 Dec 2013 04:32, "Christian Heimes" wrote:
> >
> > Hi,
> >
> > Larry has granted me a special pardon to add an outstanding fix for SSL,
> > http://bugs.python.org/issue19509 . Right now most stdlib modules
> > (ftplib, imaplib, nntplib, poplib, smtplib) neither support server name
> > indication (SNI) nor check the subject name of the peer's certificate
> > properly. The second issue is a major loop-hole because it allows
> > man-in-the-middle attack despite CERT_REQUIRED.
> >
> > With CERT_REQUIRED OpenSSL verifies that the peer's certificate is
> > directly or indirectly signed by a trusted root certification authority.
> > With Python 3.4 the ssl module is able to use/load the system's trusted
> > root certs on all major systems (Linux, Mac, BSD, Windows). On Linux and
> > BSD it requires a properly configured system openssl to locate the root
> > certs. This usually works out of the box. On Mac Apple's openssl build
> > is able to use the keychain API of OSX. I have added code for Windows'
> > system store.
> >
> > SSL socket code usually looks like this:
> >
> > context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
> > context.verify_mode = ssl.CERT_REQUIRED
> > # new, by default it loads certs trusted for Purpose.SERVER_AUTH
> > context.load_default_certs()
> >
> > sock = socket.create_connection(("example.net", 443))
> > sslsock = context.wrap_socket(sock)
> >
> > SSLContext.wrap_socket() wraps an ordinary socket into a SSLSocket. With
> > verify_mode = CERT_REQUIRED OpenSSL ensures that the peer's SSL
> > certificate is signed by a trusted root CA. In this example one very
> > important step is missing. The peer may return *ANY* signed certificate
> > for *ANY* hostname. These lines do NOT check that the certificate's
> > information match "example.net". An attacker can use any arbitrary
> > certificate (e.g. for "www.evil.net"), get it signed and abuse it for
> > MitM attacks on "mail.python.org".
> > http://docs.python.org/3/library/ssl.html#ssl.match_hostname must be
> > used to verify the cert. It's easy to forget it...
> >
> >
> > I have thought about multiple ways to fix the issue. At first I added a
> > new argument "check_hostname" to all affected modules and implemented
> > the check manually. For every module I had to modify several places for
> > SSL and STARTTLS and add / change about 10 lines. The extra lines are
> > required to properly shutdown and close the connection when the cert
> > doesn't match the hostname. I don't like the solution because it's
> > tedious. Every 3rd party author has to copy the same code, too.
> >
> > Then I came up with a better solution:
> >
> > context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
> > context.verify_mode = ssl.CERT_REQUIRED
> > context.load_default_certs()
> > context.check_hostname = True # <-- NEW
> >
> > sock = socket.create_connection(("example.net", 443))
> > # server_hostname is already used for SNI
> > sslsock = context.wrap_socket(sock, server_hostname="example.net")
> >
> >
> > This fix requires only a new SSLContext attribute and a small
> > modification to SSLSocket.do_handshake():
> >
> > if self.context.check_hostname:
> > try:
> > match_hostname(self.getpeercert(), self.server_hostname)
> > except Exception:
> > self.shutdown(_SHUT_RDWR)
> > self.close()
> > raise
> >
> >
> > Pros:
> >
> > * match_hostname() is done in one central place
> > * the cert is matched as early as possible
> > * no extra arguments for APIs, a context object is enough
> > * library developers just have to add server_hostname to get SNI and
> > hostname checks at the same time
> > * users of libraries can configure cert verification and checking on the
> > same object
> > * missing checks will not pass silently
> >
> > Cons:
> >
> > * Doesn't work with OpenSSL < 0.9.8f (released 2007) because older
> > versions lack SNI support. The ssl module raises an exception for
> > server_hostname if SNI is not supported.
> >
> >
> > The default settings for all stdlib modules will still be verify_mode =
> > CERT_NONE and check_hostname = False for maximum backward compatibility.
> > Python 3.4 comes with a new function ssl.create_default_context() that
> > returns a new context with best practice settings and loaded root CA
> > certs. The settings are TLS 1.0, no weak and insecure ciphers (no MD5,
> > no RC4), no compression (CRIME attack), CERT_REQUIRED and check_hostname
> > = True (for client side only).
> >
> > http://bugs.python.org/issue19509 has a working patch for ftplib.
> >
> > Comments?
>
> If Larry is OK with it as RM (and it sounds like he is), +1 from me as
> well.
>
--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
ht
Re: [Python-Dev] Verification of SSL cert and hostname made easy
On Sat, 30 Nov 2013 19:29:37 +0100 Christian Heimes wrote: > This fix requires only a new SSLContext attribute and a small > modification to SSLSocket.do_handshake(): > > if self.context.check_hostname: > try: > match_hostname(self.getpeercert(), self.server_hostname) > except Exception: > self.shutdown(_SHUT_RDWR) > self.close() > raise Small nit: what happens if the server_hostname is None (i.e. wasn't passed to context.wrap_socket())? > The default settings for all stdlib modules will still be verify_mode = > CERT_NONE and check_hostname = False for maximum backward compatibility. > Python 3.4 comes with a new function ssl.create_default_context() that > returns a new context with best practice settings and loaded root CA > certs. The settings are TLS 1.0, no weak and insecure ciphers (no MD5, > no RC4), no compression (CRIME attack), CERT_REQUIRED and check_hostname > = True (for client side only). Sounds fine to me, thanks. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Verification of SSL cert and hostname made easy
Am 30.11.2013 23:51, schrieb Antoine Pitrou:
> Small nit: what happens if the server_hostname is None (i.e. wasn't
> passed to context.wrap_socket())?
The code will raise an exception. My patch already implements a more
verbose ValueError that explains the cause of the problem. It's flaw in
code, that calls context.wrap_socket. Erroneous code will no longer pass
silently.
The patch also ensures a valid combination of verify_mode and
check_hostname:
>>> context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
>>> context.check_hostname = True
Traceback (most recent call last):
File "", line 1, in
ValueError: check_hostname needs a SSL context with either CERT_OPTIONAL
or CERT_REQUIRED
>>> context.verify_mode = ssl.CERT_REQUIRED
>>> context.check_hostname = True
>>> context.verify_mode = ssl.CERT_NONE
Traceback (most recent call last):
File "", line 1, in
ValueError: Cannot set verify_mode to CERT_NONE when check_hostname is
enabled.
It's only a limitation of the Python API, getpeercert() returns {} for
an unverified cert. OpenSSL can still returns the cert, though.
Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
