Bug#943666: python3: Update Python Policy for removal of the Python 2 stack

2019-10-27 Thread Neil Williams
Package: python3
Version: 3.7.5-1
Severity: normal

As discussed on IRC and alongside the post to debian-devel-announce, please
review and include this amendment to the Debian Python Policy to cover
the removal of the Python 2 stack as outlined at 
https://wiki.debian.org/Python/2Removal

https://salsa.debian.org/codehelp/python3-defaults/commit/9b1dabbdc7105486e8a2132a100066e4c225a874

Thanks.



Re: python-urllib3 1.25.6 uploaded to experimental (closes CVE-2019-11236) but fails build tests

2019-10-27 Thread Drew Parsons

On 2019-10-27 23:13, Daniele Tricoli wrote:

On Sun, Oct 13, 2019 at 10:31:31PM +0800, Drew Parsons wrote:
It conditionally works.  Using curl, I found that TLSv1_0 or TLSv1_1 
will
support a successful connection, but only if the maximum SSL_VERSION 
is
constrained to TLSv1_0 or TLSv1_1 (e.g. curl -v --tlsv1.1 --tls-max 
1.1

https://pub.orcid.org). Without the max, the connection fails:
$ curl --tlsv1.1  https://pub.orcid.org
curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert 
handshake

failure

The urllib3 failure was similar, but I do not know how to set tls-max 
with
urllib3. I could only find the option with curl.  I could set up a 
custom
HTTPAdapter as suggested at 
https://requests.readthedocs.io/en/master/user/advanced/#example-specific-ssl-version
to set ssl_version=ssl.PROTOCOL_TLSv1_1 but the ssl module doesn't 
have the
SSLVERSION_MAX_TLSv1_1 value that curl has. I could solve it with 
pycurl

using c.setopt(pycurl.SSLVERSION, pycurl.SSLVERSION_TLSv1_1 |
pycurl.SSLVERSION_MAX_TLSv1_1)


For sure I'm missing something, but why not just set TLS version?
I tried the following on both Python2 and Python3:

>>> import ssl
>>> from urllib3.poolmanager import PoolManager
>>> http = PoolManager(ssl_version=ssl.PROTOCOL_TLSv1)
>>> r = http.request('GET', 'https://pub.orcid.org')
>>> r.status
200



That's a good tip, I missed that permutation. I was originally trying to 
access using the requests module, so didn't think to do it directly with 
urllib.PoolManager





Evidently the orcid server only supports TLSv1.0 and TLSv1.1 and no 
higher
(why haven't they activated TLSv1.3 yet?!), while curl and urllib3 
without
tls-max first test TLSv1.3 and then quit without cascading downwards 
once
they receive the TLSv1.3 handshake failure.  Which is rather odd 
behaviour
when I think about it.  The whole point of supporting multiple 
protocol
versions is to try the next available version if the first one doesn't 
work.


Not an expert here, but I think fallback is not done on purpose due 
downgrade

attacks: https://en.wikipedia.org/wiki/Downgrade_attack



I see. Still an odd kind of protection though.  The attacker can just 
downgrade themselves.



I had a closer look.  The failing tests were in python2 only, coming 
from

the non-ascii (Gërman http://Königsgäßchen.de/straße and Japanese
http://ヒ:キ@ヒ.abc.ニ/ヒ?キ#ワ) unicode url tests.

...

Fixed adding python{,3}-idna on B-D. I had to add python3-idna
because the same tests were failing also on Python3 when I tested
them buinding on DoM.



Thanks for that, and thanks again for the PoolManager tip.

Drew



Re: python-urllib3 1.25.6 uploaded to experimental (closes CVE-2019-11236) but fails build tests

2019-10-27 Thread Daniele Tricoli
On Sun, Oct 13, 2019 at 10:31:31PM +0800, Drew Parsons wrote:
> It conditionally works.  Using curl, I found that TLSv1_0 or TLSv1_1 will
> support a successful connection, but only if the maximum SSL_VERSION is
> constrained to TLSv1_0 or TLSv1_1 (e.g. curl -v --tlsv1.1 --tls-max 1.1
> https://pub.orcid.org). Without the max, the connection fails:
> $ curl --tlsv1.1  https://pub.orcid.org
> curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake
> failure
> 
> The urllib3 failure was similar, but I do not know how to set tls-max with
> urllib3. I could only find the option with curl.  I could set up a custom
> HTTPAdapter as suggested at 
> https://requests.readthedocs.io/en/master/user/advanced/#example-specific-ssl-version
> to set ssl_version=ssl.PROTOCOL_TLSv1_1 but the ssl module doesn't have the
> SSLVERSION_MAX_TLSv1_1 value that curl has. I could solve it with pycurl
> using c.setopt(pycurl.SSLVERSION, pycurl.SSLVERSION_TLSv1_1 |
> pycurl.SSLVERSION_MAX_TLSv1_1)

For sure I'm missing something, but why not just set TLS version?
I tried the following on both Python2 and Python3:

>>> import ssl
>>> from urllib3.poolmanager import PoolManager
>>> http = PoolManager(ssl_version=ssl.PROTOCOL_TLSv1)
>>> r = http.request('GET', 'https://pub.orcid.org')
>>> r.status
200


> Evidently the orcid server only supports TLSv1.0 and TLSv1.1 and no higher
> (why haven't they activated TLSv1.3 yet?!), while curl and urllib3 without
> tls-max first test TLSv1.3 and then quit without cascading downwards once
> they receive the TLSv1.3 handshake failure.  Which is rather odd behaviour
> when I think about it.  The whole point of supporting multiple protocol
> versions is to try the next available version if the first one doesn't work.

Not an expert here, but I think fallback is not done on purpose due downgrade
attacks: https://en.wikipedia.org/wiki/Downgrade_attack

> I had a closer look.  The failing tests were in python2 only, coming from
> the non-ascii (Gërman http://Königsgäßchen.de/straße and Japanese
> http://ヒ:キ@ヒ.abc.ニ/ヒ?キ#ワ) unicode url tests. So from one perspective
> we don't need to worry so much about them, we could just disable them (e.g.
> prepend @onlyPy3 to test_parse_url and test_url_vulnerabilities in
> test_util.py). We'll be dropping python2 any way in the near future.
> 
> On the other hand, given the nature of the vulnerabilities and the possible
> uses of urllib3, it's probably best not to leave python2 untested,
> especially since they are known to pass on python2 anyway in the right
> conditions.  Probably there is some package that should be added to
> Build-Depends to enable python2 tests to pass, though I have no idea which
> that package might be.

Fixed adding python{,3}-idna on B-D. I had to add python3-idna
because the same tests were failing also on Python3 when I tested
them buinding on DoM.

Kind regards,

-- 
  Daniele Tricoli 'eriol'
  https://mornie.org


signature.asc
Description: PGP signature


Re: Bug#942106: (some kind of) transition: add python3.8 as a supported python3 version

2019-10-27 Thread Rebecca N. Palmer

Detailed discussion of pandas has moved to #931557 / debian-science.

Summary:
- python-pandas removal looks feasible, but there is one item that needs 
ftpmaster or release team authorization: either let pypubsub out of NEW 
(preferred), or give us permission to break tnseq-transit.


- 0.23 -> 0.25 API breakage not yet tested.  This will be the first time 
I've done a transition this size (~54 rdeps), so suggestions for good 
testing tools would be welcome.