Re: [Python-Dev] LibreSSL support

[Christian Heimes]

On 2018-01-19 15:42, Christian Heimes wrote:
> On 2018-01-19 10:43, Steve Holden wrote:
>> On Fri, Jan 19, 2018 at 12:09 AM, Nathaniel Smith > > wrote:
>>
>> On Jan 18, 2018 07:34, "Christian Heimes" > > wrote:
>>
>> On 2018-01-16 21:17, Christian Heimes wrote:
>> > FYI, master on Travis CI now builds and uses OpenSSL 1.1.0g [1]. I 
>> have
>> > created a daily cronjob to populate Travis' cache with OpenSSL 
>> builds.
>> > Until the cache is filled, Linux CI will take an extra 5 minute.
>>
>> I have messed up my initial research. :( When I was checking
>> LibreSSL
>> and OpenSSL for features, I draw a wrong conclusion. LibreSSL is
>> *not*
>> OpenSSL 1.0.2 compatible. It only implements some of the required
>> features from 1.0.2 (e.g. X509_check_hostname) but not
>> X509_VERIFY_PARAM_set1_host.
>>
>> X509_VERIFY_PARAM_set1_host() is required to perform hostname
>> verification during the TLS handshake. Without the function, I'm
>> unable
>> to fix Python's hostname matching code [1]. LibreSSL upstream knows
>> about the issue since 2016 [2]. I have opened another bug report
>> [3].
>>
>> We have two options until LibreSSL has addressed the issue:
>>
>> 1) Make the SSL module more secure, simpler and standard conform
>> 2) Support LibreSSL
>>
>>
>> ​[...]
>>
>>  
>>
>> We have *very* few people qualified to maintain the ssl module, so
>> given the new landscape I think we should focus on keeping our core
>> OpenSSL support solid and not worry about LibreSSL. If LibreSSL
>> wants to be supported as well then – like any other 2nd tier
>> platform – they need to find someone to do the work. And if people
>> are worried about supporting more diversity in SSL implementations,
>> then PEP 543 is probably the thing to focus on.
>>
>> ​Given the hard limit on resources it seems only sensible to focus on
>> the "industry standard" library​. I'm rather disappointed that LibreSSL
>> isn't a choice, but given the lack of compatibility that's hardly
>> Python's problem.
> 
> Thanks!
> 
> I'd prefer to support LibreSSL, too. Paul Kehrer from PyCA summed up the
> issue with LibreSSL nicely:
> 
>> It was marketed as an API compatible drop-in replacement and it is
> failing in that capacity. Additionally, it is missing features needed to
> improve the security and ease the maintenance burden of CPython’s dev team.
> 
> 
> Since I haven given up on LibreSSL, I spent some time and implemented
> some autoconf magic in https://github.com/python/cpython/pull/5242. It
> checks for the presence of libssl and X509_VERIFY_PARAM_set1_host()
> function family:
> 
> ...
> checking whether compiling and linking against OpenSSL works... yes
> checking for X509_VERIFY_PARAM_set1_host in libssl... yes
> ...
> 
> The ssl module will regain compatibility with LibreSSL as soon as a new
> release provides the necessary functions.

No core developer has vetoed against my proposal. I also spoke to
several members of Python Cryptographic Authority and Python Packaging
Authority. They are all in favor of my proposal, too.

There I have decided to move forward and require OpenSSL 1.0.2 API. This
means Python 3.7 temporarily suspends support for LibreSSL until
https://github.com/libressl-portable/portable/issues/381 is resolved. I
have appended a lengthy explanation to my LibreSSL ticket, too.

I also informed LibreSSL developers that Python 3.8 will most likely
require an OpenSSL 1.1 compatible API. With OpenSSL 1.0.2 support, I can
drop a considerable amount of legacy code, e.g. custom thread locking,
initialization and a bunch of shim functions.

Regards,
Christian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] LibreSSL support

[Christian Heimes]

On 2018-01-19 10:43, Steve Holden wrote:
> On Fri, Jan 19, 2018 at 12:09 AM, Nathaniel Smith  > wrote:
> 
> On Jan 18, 2018 07:34, "Christian Heimes"  > wrote:
> 
> On 2018-01-16 21:17, Christian Heimes wrote:
> > FYI, master on Travis CI now builds and uses OpenSSL 1.1.0g [1]. I 
> have
> > created a daily cronjob to populate Travis' cache with OpenSSL 
> builds.
> > Until the cache is filled, Linux CI will take an extra 5 minute.
> 
> I have messed up my initial research. :( When I was checking
> LibreSSL
> and OpenSSL for features, I draw a wrong conclusion. LibreSSL is
> *not*
> OpenSSL 1.0.2 compatible. It only implements some of the required
> features from 1.0.2 (e.g. X509_check_hostname) but not
> X509_VERIFY_PARAM_set1_host.
> 
> X509_VERIFY_PARAM_set1_host() is required to perform hostname
> verification during the TLS handshake. Without the function, I'm
> unable
> to fix Python's hostname matching code [1]. LibreSSL upstream knows
> about the issue since 2016 [2]. I have opened another bug report
> [3].
> 
> We have two options until LibreSSL has addressed the issue:
> 
> 1) Make the SSL module more secure, simpler and standard conform
> 2) Support LibreSSL
> 
> 
> ​[...]
> 
>  
> 
> We have *very* few people qualified to maintain the ssl module, so
> given the new landscape I think we should focus on keeping our core
> OpenSSL support solid and not worry about LibreSSL. If LibreSSL
> wants to be supported as well then – like any other 2nd tier
> platform – they need to find someone to do the work. And if people
> are worried about supporting more diversity in SSL implementations,
> then PEP 543 is probably the thing to focus on.
> 
> ​Given the hard limit on resources it seems only sensible to focus on
> the "industry standard" library​. I'm rather disappointed that LibreSSL
> isn't a choice, but given the lack of compatibility that's hardly
> Python's problem.

Thanks!

I'd prefer to support LibreSSL, too. Paul Kehrer from PyCA summed up the
issue with LibreSSL nicely:

> It was marketed as an API compatible drop-in replacement and it is
failing in that capacity. Additionally, it is missing features needed to
improve the security and ease the maintenance burden of CPython’s dev team.


Since I haven given up on LibreSSL, I spent some time and implemented
some autoconf magic in https://github.com/python/cpython/pull/5242. It
checks for the presence of libssl and X509_VERIFY_PARAM_set1_host()
function family:

...
checking whether compiling and linking against OpenSSL works... yes
checking for X509_VERIFY_PARAM_set1_host in libssl... yes
...

The ssl module will regain compatibility with LibreSSL as soon as a new
release provides the necessary functions.

Christian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] LibreSSL support

[Steve Holden]

On Fri, Jan 19, 2018 at 12:09 AM, Nathaniel Smith  wrote:

> On Jan 18, 2018 07:34, "Christian Heimes"  wrote:
>
> On 2018-01-16 21:17, Christian Heimes wrote:
> > FYI, master on Travis CI now builds and uses OpenSSL 1.1.0g [1]. I have
> > created a daily cronjob to populate Travis' cache with OpenSSL builds.
> > Until the cache is filled, Linux CI will take an extra 5 minute.
>
> I have messed up my initial research. :( When I was checking LibreSSL
> and OpenSSL for features, I draw a wrong conclusion. LibreSSL is *not*
> OpenSSL 1.0.2 compatible. It only implements some of the required
> features from 1.0.2 (e.g. X509_check_hostname) but not
> X509_VERIFY_PARAM_set1_host.
>
> X509_VERIFY_PARAM_set1_host() is required to perform hostname
> verification during the TLS handshake. Without the function, I'm unable
> to fix Python's hostname matching code [1]. LibreSSL upstream knows
> about the issue since 2016 [2]. I have opened another bug report [3].
>
> We have two options until LibreSSL has addressed the issue:
>
> 1) Make the SSL module more secure, simpler and standard conform
> 2) Support LibreSSL
>
>
> ​[...]
>


> We have *very* few people qualified to maintain the ssl module, so given
> the new landscape I think we should focus on keeping our core OpenSSL
> support solid and not worry about LibreSSL. If LibreSSL wants to be
> supported as well then – like any other 2nd tier platform – they need to
> find someone to do the work. And if people are worried about supporting
> more diversity in SSL implementations, then PEP 543 is probably the thing
> to focus on.
>
> ​Given the hard limit on resources it seems only sensible to focus on the
"industry standard" library​. I'm rather disappointed that LibreSSL isn't a
choice, but given the lack of compatibility that's hardly Python's problem.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] LibreSSL support

[Nathaniel Smith]

On Jan 18, 2018 07:34, "Christian Heimes"  wrote:

On 2018-01-16 21:17, Christian Heimes wrote:
> FYI, master on Travis CI now builds and uses OpenSSL 1.1.0g [1]. I have
> created a daily cronjob to populate Travis' cache with OpenSSL builds.
> Until the cache is filled, Linux CI will take an extra 5 minute.

I have messed up my initial research. :( When I was checking LibreSSL
and OpenSSL for features, I draw a wrong conclusion. LibreSSL is *not*
OpenSSL 1.0.2 compatible. It only implements some of the required
features from 1.0.2 (e.g. X509_check_hostname) but not
X509_VERIFY_PARAM_set1_host.

X509_VERIFY_PARAM_set1_host() is required to perform hostname
verification during the TLS handshake. Without the function, I'm unable
to fix Python's hostname matching code [1]. LibreSSL upstream knows
about the issue since 2016 [2]. I have opened another bug report [3].

We have two options until LibreSSL has addressed the issue:

1) Make the SSL module more secure, simpler and standard conform
2) Support LibreSSL


There are tons of different SSL libraries out there that we could
theoretically support, but don't. IIUC, the reasons we started supporting
LibreSSL in the first place were:

- it claimed to be OpenSSL compatible, so supporting it is supposed to be
"free"

- when it started (just after heartbleed), there was a lot of uncertainty
about whether OpenSSL would remain a viable option, and LibreSSL looked
like it might become the new de facto standard.

Now it's a few years later, and things have turned out differently: they
aren't compatible in practice, and OpenSSL has turned things around so that
it's clearly ahead of LibreSSL technically (cleaner API, new features like
TLS 1.3, ...), and in terms of developer momentum.

We have *very* few people qualified to maintain the ssl module, so given
the new landscape I think we should focus on keeping our core OpenSSL
support solid and not worry about LibreSSL. If LibreSSL wants to be
supported as well then – like any other 2nd tier platform – they need to
find someone to do the work. And if people are worried about supporting
more diversity in SSL implementations, then PEP 543 is probably the thing
to focus on.

-n
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] LibreSSL support

[Christian Heimes]

On 2018-01-18 21:49, Chris Jerdonek wrote:
> 
> On Thu, Jan 18, 2018 at 7:34 AM Christian Heimes  > wrote:
> 
> On 2018-01-16 21:17, Christian Heimes wrote:
> We have two options until LibreSSL has addressed the issue:
> 
> 1) Make the SSL module more secure, simpler and standard conform
> 2) Support LibreSSL
> 
> I started a vote on Twitter [4]. So far most people prefer security.
> 
> 
> It’s not exactly the most balanced (neutral) presentation of a ballot
> question though. :)

It's more venting than voting :)

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] LibreSSL support

[Chris Jerdonek]

On Thu, Jan 18, 2018 at 7:34 AM Christian Heimes 
wrote:

> On 2018-01-16 21:17, Christian Heimes wrote:
> We have two options until LibreSSL has addressed the issue:
>
> 1) Make the SSL module more secure, simpler and standard conform
> 2) Support LibreSSL
>
> I started a vote on Twitter [4]. So far most people prefer security.


It’s not exactly the most balanced (neutral) presentation of a ballot
question though. :)

—Chris


>
> Christian
>
> [1] https://bugs.python.org/issue31399
> [2] https://github.com/pyca/cryptography/issues/3247
> [3] https://github.com/libressl-portable/portable/issues/381
> [4] https://twitter.com/reaperhulk/status/953991843565490176
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/chris.jerdonek%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] LibreSSL support

[Christian Heimes]

On 2018-01-18 20:54, Wes Turner wrote:
> LibreSSL is not a pressing need for me; but fallback to the existing
> insecure check if LibreSSL is present shouldn't be too difficult?

Please give it a try and report back. Patches welcome :)

Christian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] LibreSSL support

[Wes Turner]

LibreSSL is not a pressing need for me; but fallback to the existing
insecure check if LibreSSL is present shouldn't be too difficult?

On Thursday, January 18, 2018, Christian Heimes 
wrote:

> On 2018-01-18 19:42, Wes Turner wrote:
> > Is there a build flag or a ./configure-time autodetection that would
> > allow for supporting LibreSSL while they port
> X509_VERIFY_PARAM_set1_host?
>
> X509_VERIFY_PARAM_set1_host() is a fundamental and essential piece in
> the new hostname verification code. I cannot replace
> ssl.match_hostname() easily without the API. There might be a way to add
> a callback, but it would take a couple of days of R to implement it.
> It won't be finished for beta1 feature freeze.
>
> Christian
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] LibreSSL support

[Christian Heimes]

On 2018-01-18 19:42, Wes Turner wrote:
> Is there a build flag or a ./configure-time autodetection that would
> allow for supporting LibreSSL while they port X509_VERIFY_PARAM_set1_host?

X509_VERIFY_PARAM_set1_host() is a fundamental and essential piece in
the new hostname verification code. I cannot replace
ssl.match_hostname() easily without the API. There might be a way to add
a callback, but it would take a couple of days of R to implement it.
It won't be finished for beta1 feature freeze.

Christian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] LibreSSL support

[Wes Turner]

Is there a build flag or a ./configure-time autodetection that would allow
for supporting LibreSSL while they port X509_VERIFY_PARAM_set1_host?

On Thursday, January 18, 2018, Christian Heimes 
wrote:

> On 2018-01-16 21:17, Christian Heimes wrote:
> > FYI, master on Travis CI now builds and uses OpenSSL 1.1.0g [1]. I have
> > created a daily cronjob to populate Travis' cache with OpenSSL builds.
> > Until the cache is filled, Linux CI will take an extra 5 minute.
>
> I have messed up my initial research. :( When I was checking LibreSSL
> and OpenSSL for features, I draw a wrong conclusion. LibreSSL is *not*
> OpenSSL 1.0.2 compatible. It only implements some of the required
> features from 1.0.2 (e.g. X509_check_hostname) but not
> X509_VERIFY_PARAM_set1_host.
>
> X509_VERIFY_PARAM_set1_host() is required to perform hostname
> verification during the TLS handshake. Without the function, I'm unable
> to fix Python's hostname matching code [1]. LibreSSL upstream knows
> about the issue since 2016 [2]. I have opened another bug report [3].
>
> We have two options until LibreSSL has addressed the issue:
>
> 1) Make the SSL module more secure, simpler and standard conform
> 2) Support LibreSSL
>
> I started a vote on Twitter [4]. So far most people prefer security.
>
> Christian
>
> [1] https://bugs.python.org/issue31399
> [2] https://github.com/pyca/cryptography/issues/3247
> [3] https://github.com/libressl-portable/portable/issues/381
> [4] https://twitter.com/reaperhulk/status/953991843565490176
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com