Re: [Distutils] Announcement: TLSv1.2 will become mandatory in thefuture

[Steve Dower]

"I don’t think it’s a particularly big deal to tie the tls module to the Python 
lifecycle though"

I'd hope that the API of this module is stable enough and the native part of 
the implementation is OS-specific enough that we may not even need to update it 
that often. (I'm advocating very strongly for just using the OS APIs to 
implement it, and those don't change often enough for us to need to worry.)

The Linux builds can link to OpenSSL, but there shouldn't be anything requiring 
OpenSSL for this module, so the update timeframe is totally different. But I've 
now joined security-sig, which is where the discussion seems to be, so I'll 
stop designing things here :)

Top-posted from my Windows Phone

-Original Message-
From: "Donald Stufft" 
Sent: ‎1/‎11/‎2017 19:48
To: "Nick Coghlan" 
Cc: "DistUtils mailing list" 
Subject: Re: [Distutils] Announcement: TLSv1.2 will become mandatory in 
thefuture



On Jan 11, 2017, at 10:40 PM, Nick Coghlan  wrote:


On 12 January 2017 at 13:00, Donald Stufft  wrote:

This doesn’t work well because it’s not something that pip is going to be
able to upgrade on Windows, because the .so will be locked when pip imports
it on Windows and we won’t be able to uninstall it to do an upgrade. We had
to disable the automatic use of pyOpenSSL for this reason too. The only C
stuff that pip can reliably use is the standard library.


Ugh, I'd completely forgotten about that limitation of Windows filesystems.

And the main alternatives I can think of involve copying files around
as pip starts up, which would be unacceptably slow for a command line
app :(




I don’t think it’s a particularly big deal to tie the tls module to the Python 
lifecycle though, we’ve got a precident for PEPs that backport important 
security critical stuff and most things are presumably going to be things that 
we don’t really even need a backport or a PEP for (I’m thinking things like 
ciphers and such). Particularly if this new thing is documented up front 
clearly what things you can depend on for compatibility (api and such) and what 
things can change in minor releases (keeping up with the security joneses 
stuff).


I think the big thing that really killed the ssl module for so long in Python 
was the 2.x vs 3.x split with 2.7 living for a _very_ long time, and then no 
culture of back porting security important changes to it.



—
Donald Stufft___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Announcement: TLSv1.2 will become mandatory in the future

[Nick Coghlan]

On 12 January 2017 at 14:12, Glyph Lefkowitz  wrote:
> It's possible for Pip to notice that it wants to replace a particular file;
> you can "unlock" it by moving it aside.
>
> https://serverfault.com/a/503769

Very interesting, especially as the way CPython loads extension
modules means that the renaming trick should be safe (even if you
"reload" an extension module, or delete it from sys.modules and import
it again, CPython doesn't reload the underlying DLL).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Announcement: TLSv1.2 will become mandatory in the future

[Nick Coghlan]

On 12 January 2017 at 13:47, Donald Stufft  wrote:
> I don’t think it’s a particularly big deal to tie the tls module to the
> Python lifecycle though, we’ve got a precident for PEPs that backport
> important security critical stuff and most things are presumably going to be
> things that we don’t really even need a backport or a PEP for (I’m thinking
> things like ciphers and such). Particularly if this new thing is documented
> up front clearly what things you can depend on for compatibility (api and
> such) and what things can change in minor releases (keeping up with the
> security joneses stuff).
>
> I think the big thing that really killed the ssl module for so long in
> Python was the 2.x vs 3.x split with 2.7 living for a _very_ long time, and
> then no culture of back porting security important changes to it.

True, it took ~4 years for 2.7 to really fall unacceptably far behind
the state of the art, and even then it was as much about the lack of
SNI support as it was anything else.

If a new tls module started out with an API management policy that
allowed for new constants and for changes to the default security
settings in maintenance releases, then it would likely only need two
PEPs to define an effective rollout plan:

- one to add it to 3.7+
- one to backport the initial version to 2.7.x (and maybe the other
actively supported 3.x branches)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Announcement: TLSv1.2 will become mandatory in the future

[Glyph Lefkowitz]

> On Jan 11, 2017, at 7:40 PM, Nick Coghlan  wrote:
> 
> On 12 January 2017 at 13:00, Donald Stufft  wrote:
>> This doesn’t work well because it’s not something that pip is going to be
>> able to upgrade on Windows, because the .so will be locked when pip imports
>> it on Windows and we won’t be able to uninstall it to do an upgrade. We had
>> to disable the automatic use of pyOpenSSL for this reason too. The only C
>> stuff that pip can reliably use is the standard library.
> 
> Ugh, I'd completely forgotten about that limitation of Windows filesystems.
> 
> And the main alternatives I can think of involve copying files around
> as pip starts up, which would be unacceptably slow for a command line
> app :(


It's possible for Pip to notice that it wants to replace a particular file; you 
can "unlock" it by moving it aside.

https://serverfault.com/a/503769 
-glyph___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Announcement: TLSv1.2 will become mandatory in the future

[Donald Stufft]

> On Jan 11, 2017, at 10:40 PM, Nick Coghlan  wrote:
> 
> On 12 January 2017 at 13:00, Donald Stufft  wrote:
>> This doesn’t work well because it’s not something that pip is going to be
>> able to upgrade on Windows, because the .so will be locked when pip imports
>> it on Windows and we won’t be able to uninstall it to do an upgrade. We had
>> to disable the automatic use of pyOpenSSL for this reason too. The only C
>> stuff that pip can reliably use is the standard library.
> 
> Ugh, I'd completely forgotten about that limitation of Windows filesystems.
> 
> And the main alternatives I can think of involve copying files around
> as pip starts up, which would be unacceptably slow for a command line
> app :(
> 

I don’t think it’s a particularly big deal to tie the tls module to the Python 
lifecycle though, we’ve got a precident for PEPs that backport important 
security critical stuff and most things are presumably going to be things that 
we don’t really even need a backport or a PEP for (I’m thinking things like 
ciphers and such). Particularly if this new thing is documented up front 
clearly what things you can depend on for compatibility (api and such) and what 
things can change in minor releases (keeping up with the security joneses 
stuff).

I think the big thing that really killed the ssl module for so long in Python 
was the 2.x vs 3.x split with 2.7 living for a _very_ long time, and then no 
culture of back porting security important changes to it.


—
Donald Stufft



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Announcement: TLSv1.2 will become mandatory in the future

[Nick Coghlan]

On 12 January 2017 at 13:00, Donald Stufft  wrote:
> This doesn’t work well because it’s not something that pip is going to be
> able to upgrade on Windows, because the .so will be locked when pip imports
> it on Windows and we won’t be able to uninstall it to do an upgrade. We had
> to disable the automatic use of pyOpenSSL for this reason too. The only C
> stuff that pip can reliably use is the standard library.

Ugh, I'd completely forgotten about that limitation of Windows filesystems.

And the main alternatives I can think of involve copying files around
as pip starts up, which would be unacceptably slow for a command line
app :(

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Announcement: TLSv1.2 will become mandatory in the future

[Donald Stufft]

> On Jan 11, 2017, at 9:58 PM, Nick Coghlan  wrote:
> 
> On 12 January 2017 at 04:26, Brett Cannon  wrote:
>> 
>> On Tue, 10 Jan 2017 at 12:51 Donald Stufft  wrote:
>>> 
>>> [SNIP]
>>> 
>>> 
>>> It would be really nice if we could deprecate `ssl` (which has a bunch of
>>> OpenSSL specific stuff in it) and add a new `tls` module that served as an
>>> implementation agnostic library that would use OpenSSL on *nix,
>>> SecureTransport on macOS, and SChannel on Windows. However, in the mean time
>>> there are some folks poking to see about making something pip suitable that
>>> will enable us to use SecureTransport at least.
>> 
>> 
>> I know both Cory Benfield and Christian Heimes brought this up briefly at
>> the PyCon US 2016 language summit at the end of their SSL discussion, but I
>> don't think it went anywhere because there was some other discussion that
>> dominated the end of their talk (I've now tweeted at them about this
>> discussion).
>> 
>> I know Steve has also said he would love to see a agnostic TLS library so
>> that Windows' built-in libraries for this stuff could be directly used. With
>> the predicament this is going to put us in I think it makes it very prudent
>> to create a tls module for the stdlib.
> 
> Logistically, something I think we should explore for such a module is
> using the same ensuretls/tls split that we did for ensurepip/pip. That
> way it can be more readily updated in line with the evolution of
> network security standards and operating system crpytographic APIs,
> rather than being locked into being updated in line with the evolution
> of the Python language definition.
> 


This doesn’t work well because it’s not something that pip is going to be able 
to upgrade on Windows, because the .so will be locked when pip imports it on 
Windows and we won’t be able to uninstall it to do an upgrade. We had to 
disable the automatic use of pyOpenSSL for this reason too. The only C stuff 
that pip can reliably use is the standard library.

—
Donald Stufft



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Announcement: TLSv1.2 will become mandatory in the future

[Nick Coghlan]

On 12 January 2017 at 04:26, Brett Cannon  wrote:
>
> On Tue, 10 Jan 2017 at 12:51 Donald Stufft  wrote:
>>
>> [SNIP]
>>
>>
>> It would be really nice if we could deprecate `ssl` (which has a bunch of
>> OpenSSL specific stuff in it) and add a new `tls` module that served as an
>> implementation agnostic library that would use OpenSSL on *nix,
>> SecureTransport on macOS, and SChannel on Windows. However, in the mean time
>> there are some folks poking to see about making something pip suitable that
>> will enable us to use SecureTransport at least.
>
>
> I know both Cory Benfield and Christian Heimes brought this up briefly at
> the PyCon US 2016 language summit at the end of their SSL discussion, but I
> don't think it went anywhere because there was some other discussion that
> dominated the end of their talk (I've now tweeted at them about this
> discussion).
>
> I know Steve has also said he would love to see a agnostic TLS library so
> that Windows' built-in libraries for this stuff could be directly used. With
> the predicament this is going to put us in I think it makes it very prudent
> to create a tls module for the stdlib.

Logistically, something I think we should explore for such a module is
using the same ensuretls/tls split that we did for ensurepip/pip. That
way it can be more readily updated in line with the evolution of
network security standards and operating system crpytographic APIs,
rather than being locked into being updated in line with the evolution
of the Python language definition.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Announcement: TLSv1.2 will become mandatory in the future

[Ethan Furman]

On 01/11/2017 10:26 AM, Brett Cannon wrote:


I know both Cory Benfield and Christian Heimes brought this up briefly
 at the PyCon US 2016 language summit at the end of their SSL discussion,
but I don't think it went anywhere because there was some other
 discussion that dominated the end of their talk (I've now tweeted at
 them about this discussion).

I know Steve has also said he would love to see a agnostic TLS library so
 that Windows' built-in libraries for this stuff could be directly used.
 With the predicament this is going to put us in I think it makes it very
 prudent to create a tls module for the stdlib.


A thread about a new tls module has been started on the security-sig [1].

--
~Ethan~

[1] security-...@python.org
https://mail.python.org/pipermail/security-sig/
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Announcement: TLSv1.2 will become mandatory in the future

[Brett Cannon]

On Tue, 10 Jan 2017 at 12:51 Donald Stufft  wrote:

> [SNIP]
>
>
> It would be really nice if we could deprecate `ssl` (which has a bunch of
> OpenSSL specific stuff in it) and add a new `tls` module that served as an
> implementation agnostic library that would use OpenSSL on *nix,
> SecureTransport on macOS, and SChannel on Windows. However, in the mean
> time there are some folks poking to see about making something pip suitable
> that will enable us to use SecureTransport at least.
>

I know both Cory Benfield and Christian Heimes brought this up briefly at
the PyCon US 2016 language summit at the end of their SSL discussion, but I
don't think it went anywhere because there was some other discussion that
dominated the end of their talk (I've now tweeted at them about this
discussion).

I know Steve has also said he would love to see a agnostic TLS library so
that Windows' built-in libraries for this stuff could be directly used.
With the predicament this is going to put us in I think it makes it very
prudent to create a tls module for the stdlib.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Announcement: TLSv1.2 will become mandatory in the future

[Eric V. Smith]

On 1/11/2017 8:09 AM, Ronald Oussoren wrote:



On 10 Jan 2017, at 21:47, Ned Deily  wrote:



Annoyingly with OpenSSL on OSX you have to options: either use an up-to-date 
release or have OpenSSL use the system CA trust store, but not both.  Sigh…


It would be nice if someone would do the work to figure out whether it is 
feasible to use Apple's own Crypto and TLS API's as apparently libcurl does.


This should be possible, although I don’t know if the relevant Apple API’s are 
safe to use after forking (which has bitten us in the past for other Apple 
APIs, such as the _scproxy extension that detects proxies on OSX).

Sadly enough I don’t really have time to look into this.


While I'm sure this isn't a newbie "easy" issue, maybe open an issue for 
it on the bug tracker and someone will find it and work on it? I'd open 
it myself, but I don't know enough about it to phrase the problem correctly!


Eric.


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig