Re: python-urllib3 1.25.6 uploaded to experimental (closes CVE-2019-11236) but fails build tests

2019-10-29 Thread Michael Kesper
Hi all,

On 29.10.19 14:15, Jeremy Stanley wrote:
> On 2019-10-29 13:29:02 +0100 (+0100), Michael Kesper wrote:
>> On 27.10.19 17:27, Drew Parsons wrote:
>>> On 2019-10-27 23:13, Daniele Tricoli wrote:
> [...]
 Not an expert here, but I think fallback is not done on
 purpose due downgrade attacks:
 https://en.wikipedia.org/wiki/Downgrade_attack
>>>
>>> I see. Still an odd kind of protection though.  The attacker can
>>> just downgrade themselves.
>>
>> No. A sensible server will not talk to you if your requested SSL
>> version is too low. pub.orcid.org seems to use absolutely outdated
>> and insecure software versions.
> 
> Well, downgrade attacks aren't usually a two-party scenario. The
> risk with a downgrade attack is when a victim client attempts
> communication with some server, and a third-party attacker tampers
> with the communication between the client and server sufficiently to
> cause protocol negotiation to fall back to an old enough version
> that the attacker can then exploit known flaws to decrypt and/or
> proxy ("man in the middle") that communication. Having both the
> client and the server be unwilling to use susceptible older protocol
> versions helps thwart this attack vector.

Ah, you're right.
So the right fix would be to remove this possibility from urllib3.
This, however, would break applications that need to use these insecure 
connections.
At the least, I don't think build tests should fail if software
refuses to use deprecated and insecure connections.

Bye
Michael



signature.asc
Description: OpenPGP digital signature


Re: python-urllib3 1.25.6 uploaded to experimental (closes CVE-2019-11236) but fails build tests

2019-10-29 Thread Jeremy Stanley
On 2019-10-29 13:29:02 +0100 (+0100), Michael Kesper wrote:
> On 27.10.19 17:27, Drew Parsons wrote:
> > On 2019-10-27 23:13, Daniele Tricoli wrote:
[...]
> > > Not an expert here, but I think fallback is not done on
> > > purpose due downgrade attacks:
> > > https://en.wikipedia.org/wiki/Downgrade_attack
> > 
> > I see. Still an odd kind of protection though.  The attacker can
> > just downgrade themselves.
> 
> No. A sensible server will not talk to you if your requested SSL
> version is too low. pub.orcid.org seems to use absolutely outdated
> and insecure software versions.

Well, downgrade attacks aren't usually a two-party scenario. The
risk with a downgrade attack is when a victim client attempts
communication with some server, and a third-party attacker tampers
with the communication between the client and server sufficiently to
cause protocol negotiation to fall back to an old enough version
that the attacker can then exploit known flaws to decrypt and/or
proxy ("man in the middle") that communication. Having both the
client and the server be unwilling to use susceptible older protocol
versions helps thwart this attack vector.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature


Re: python-urllib3 1.25.6 uploaded to experimental (closes CVE-2019-11236) but fails build tests

2019-10-29 Thread Ondrej Novy
Hi,

út 29. 10. 2019 v 13:29 odesílatel Michael Kesper 
napsal:

> > I see. Still an odd kind of protection though.  The attacker can just
> downgrade themselves.
>
> No. A sensible server will not talk to you if your requested SSL version
> is too low.
> pub.orcid.org seems to use absolutely outdated and insecure software
> versions.
>

right. If you want good security, you need to limit TLS version on both
side (client-urlib3 and server-whatever). Than downgrade attack is not
possible.

-- 
Best regards
 Ondřej Nový


Re: python-urllib3 1.25.6 uploaded to experimental (closes CVE-2019-11236) but fails build tests

2019-10-29 Thread Michael Kesper
Hi all,

On 27.10.19 17:27, Drew Parsons wrote:
> On 2019-10-27 23:13, Daniele Tricoli wrote:
>> On Sun, Oct 13, 2019 at 10:31:31PM +0800, Drew Parsons wrote:
>>> It conditionally works.  Using curl, I found that TLSv1_0 or TLSv1_1 will
>>> support a successful connection, but only if the maximum SSL_VERSION is
>>> constrained to TLSv1_0 or TLSv1_1 (e.g. curl -v --tlsv1.1 --tls-max 1.1
>>> https://pub.orcid.org). Without the max, the connection fails:
>>> $ curl --tlsv1.1  https://pub.orcid.org
>>> curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake
>>> failure
>>>
>>> The urllib3 failure was similar, but I do not know how to set tls-max with
>>> urllib3. I could only find the option with curl.  I could set up a custom
>>> HTTPAdapter as suggested at 
>>> https://requests.readthedocs.io/en/master/user/advanced/#example-specific-ssl-version
>>> to set ssl_version=ssl.PROTOCOL_TLSv1_1 but the ssl module doesn't have the
>>> SSLVERSION_MAX_TLSv1_1 value that curl has. I could solve it with pycurl
>>> using c.setopt(pycurl.SSLVERSION, pycurl.SSLVERSION_TLSv1_1 |
>>> pycurl.SSLVERSION_MAX_TLSv1_1)
>>
>> For sure I'm missing something, but why not just set TLS version?
>> I tried the following on both Python2 and Python3:
>>
>>     >>> import ssl
>>     >>> from urllib3.poolmanager import PoolManager
>>     >>> http = PoolManager(ssl_version=ssl.PROTOCOL_TLSv1)
>>     >>> r = http.request('GET', 'https://pub.orcid.org')
>>     >>> r.status
>>     200
> 
> 
> That's a good tip, I missed that permutation. I was originally trying to 
> access using the requests module, so didn't think to do it directly with 
> urllib.PoolManager
> 
> 
>>
>>> Evidently the orcid server only supports TLSv1.0 and TLSv1.1 and no higher
>>> (why haven't they activated TLSv1.3 yet?!), while curl and urllib3 without
>>> tls-max first test TLSv1.3 and then quit without cascading downwards once
>>> they receive the TLSv1.3 handshake failure.  Which is rather odd behaviour
>>> when I think about it.  The whole point of supporting multiple protocol
>>> versions is to try the next available version if the first one doesn't work.
>>
>> Not an expert here, but I think fallback is not done on purpose due downgrade
>> attacks: https://en.wikipedia.org/wiki/Downgrade_attack
> 
> 
> I see. Still an odd kind of protection though.  The attacker can just 
> downgrade themselves.

No. A sensible server will not talk to you if your requested SSL version is too 
low.
pub.orcid.org seems to use absolutely outdated and insecure software versions.

Bye
Michael



signature.asc
Description: OpenPGP digital signature


Re: python-urllib3 1.25.6 uploaded to experimental (closes CVE-2019-11236) but fails build tests

2019-10-27 Thread Drew Parsons

On 2019-10-27 23:13, Daniele Tricoli wrote:

On Sun, Oct 13, 2019 at 10:31:31PM +0800, Drew Parsons wrote:
It conditionally works.  Using curl, I found that TLSv1_0 or TLSv1_1 
will
support a successful connection, but only if the maximum SSL_VERSION 
is
constrained to TLSv1_0 or TLSv1_1 (e.g. curl -v --tlsv1.1 --tls-max 
1.1

https://pub.orcid.org). Without the max, the connection fails:
$ curl --tlsv1.1  https://pub.orcid.org
curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert 
handshake

failure

The urllib3 failure was similar, but I do not know how to set tls-max 
with
urllib3. I could only find the option with curl.  I could set up a 
custom
HTTPAdapter as suggested at 
https://requests.readthedocs.io/en/master/user/advanced/#example-specific-ssl-version
to set ssl_version=ssl.PROTOCOL_TLSv1_1 but the ssl module doesn't 
have the
SSLVERSION_MAX_TLSv1_1 value that curl has. I could solve it with 
pycurl

using c.setopt(pycurl.SSLVERSION, pycurl.SSLVERSION_TLSv1_1 |
pycurl.SSLVERSION_MAX_TLSv1_1)


For sure I'm missing something, but why not just set TLS version?
I tried the following on both Python2 and Python3:

>>> import ssl
>>> from urllib3.poolmanager import PoolManager
>>> http = PoolManager(ssl_version=ssl.PROTOCOL_TLSv1)
>>> r = http.request('GET', 'https://pub.orcid.org')
>>> r.status
200



That's a good tip, I missed that permutation. I was originally trying to 
access using the requests module, so didn't think to do it directly with 
urllib.PoolManager





Evidently the orcid server only supports TLSv1.0 and TLSv1.1 and no 
higher
(why haven't they activated TLSv1.3 yet?!), while curl and urllib3 
without
tls-max first test TLSv1.3 and then quit without cascading downwards 
once
they receive the TLSv1.3 handshake failure.  Which is rather odd 
behaviour
when I think about it.  The whole point of supporting multiple 
protocol
versions is to try the next available version if the first one doesn't 
work.


Not an expert here, but I think fallback is not done on purpose due 
downgrade

attacks: https://en.wikipedia.org/wiki/Downgrade_attack



I see. Still an odd kind of protection though.  The attacker can just 
downgrade themselves.



I had a closer look.  The failing tests were in python2 only, coming 
from

the non-ascii (Gërman http://Königsgäßchen.de/straße and Japanese
http://ヒ:キ@ヒ.abc.ニ/ヒ?キ#ワ) unicode url tests.

...

Fixed adding python{,3}-idna on B-D. I had to add python3-idna
because the same tests were failing also on Python3 when I tested
them buinding on DoM.



Thanks for that, and thanks again for the PoolManager tip.

Drew



Re: python-urllib3 1.25.6 uploaded to experimental (closes CVE-2019-11236) but fails build tests

2019-10-27 Thread Daniele Tricoli
On Sun, Oct 13, 2019 at 10:31:31PM +0800, Drew Parsons wrote:
> It conditionally works.  Using curl, I found that TLSv1_0 or TLSv1_1 will
> support a successful connection, but only if the maximum SSL_VERSION is
> constrained to TLSv1_0 or TLSv1_1 (e.g. curl -v --tlsv1.1 --tls-max 1.1
> https://pub.orcid.org). Without the max, the connection fails:
> $ curl --tlsv1.1  https://pub.orcid.org
> curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake
> failure
> 
> The urllib3 failure was similar, but I do not know how to set tls-max with
> urllib3. I could only find the option with curl.  I could set up a custom
> HTTPAdapter as suggested at 
> https://requests.readthedocs.io/en/master/user/advanced/#example-specific-ssl-version
> to set ssl_version=ssl.PROTOCOL_TLSv1_1 but the ssl module doesn't have the
> SSLVERSION_MAX_TLSv1_1 value that curl has. I could solve it with pycurl
> using c.setopt(pycurl.SSLVERSION, pycurl.SSLVERSION_TLSv1_1 |
> pycurl.SSLVERSION_MAX_TLSv1_1)

For sure I'm missing something, but why not just set TLS version?
I tried the following on both Python2 and Python3:

>>> import ssl
>>> from urllib3.poolmanager import PoolManager
>>> http = PoolManager(ssl_version=ssl.PROTOCOL_TLSv1)
>>> r = http.request('GET', 'https://pub.orcid.org')
>>> r.status
200


> Evidently the orcid server only supports TLSv1.0 and TLSv1.1 and no higher
> (why haven't they activated TLSv1.3 yet?!), while curl and urllib3 without
> tls-max first test TLSv1.3 and then quit without cascading downwards once
> they receive the TLSv1.3 handshake failure.  Which is rather odd behaviour
> when I think about it.  The whole point of supporting multiple protocol
> versions is to try the next available version if the first one doesn't work.

Not an expert here, but I think fallback is not done on purpose due downgrade
attacks: https://en.wikipedia.org/wiki/Downgrade_attack

> I had a closer look.  The failing tests were in python2 only, coming from
> the non-ascii (Gërman http://Königsgäßchen.de/straße and Japanese
> http://ヒ:キ@ヒ.abc.ニ/ヒ?キ#ワ) unicode url tests. So from one perspective
> we don't need to worry so much about them, we could just disable them (e.g.
> prepend @onlyPy3 to test_parse_url and test_url_vulnerabilities in
> test_util.py). We'll be dropping python2 any way in the near future.
> 
> On the other hand, given the nature of the vulnerabilities and the possible
> uses of urllib3, it's probably best not to leave python2 untested,
> especially since they are known to pass on python2 anyway in the right
> conditions.  Probably there is some package that should be added to
> Build-Depends to enable python2 tests to pass, though I have no idea which
> that package might be.

Fixed adding python{,3}-idna on B-D. I had to add python3-idna
because the same tests were failing also on Python3 when I tested
them buinding on DoM.

Kind regards,

-- 
  Daniele Tricoli 'eriol'
  https://mornie.org


signature.asc
Description: PGP signature


Re: python-urllib3 1.25.6 uploaded to experimental (closes CVE-2019-11236) but fails build tests

2019-10-13 Thread Drew Parsons

Daniele wrote:
I hope to have the time to investigate also this: 
urllib3/contrib/pyopenssl.py
contains code to have SSL with SNI_-support for Python 2 and it depends 
on
pyOpenSSL, cryptography and idna. Maybe looking at them can give us 
more clues.


Also, could you see if using Python3 the connection to 
https://pub.orcid.org work?


It conditionally works.  Using curl, I found that TLSv1_0 or TLSv1_1 
will support a successful connection, but only if the maximum 
SSL_VERSION is constrained to TLSv1_0 or TLSv1_1 (e.g. curl -v --tlsv1.1 
--tls-max 1.1  https://pub.orcid.org). Without the max, the connection 
fails:

$ curl --tlsv1.1  https://pub.orcid.org
curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert 
handshake failure


The urllib3 failure was similar, but I do not know how to set tls-max 
with urllib3. I could only find the option with curl.  I could set up a 
custom HTTPAdapter as suggested at 
https://requests.readthedocs.io/en/master/user/advanced/#example-specific-ssl-version 
to set ssl_version=ssl.PROTOCOL_TLSv1_1 but the ssl module doesn't have 
the SSLVERSION_MAX_TLSv1_1 value that curl has. I could solve it with 
pycurl using c.setopt(pycurl.SSLVERSION, pycurl.SSLVERSION_TLSv1_1 | 
pycurl.SSLVERSION_MAX_TLSv1_1)


Evidently the orcid server only supports TLSv1.0 and TLSv1.1 and no 
higher (why haven't they activated TLSv1.3 yet?!), while curl and 
urllib3 without tls-max first test TLSv1.3 and then quit without 
cascading downwards once they receive the TLSv1.3 handshake failure.  
Which is rather odd behaviour when I think about it.  The whole point of 
supporting multiple protocol versions is to try the next available 
version if the first one doesn't work.




Th package build was successful on my system but gives build-time 
errors in

chroot (on buildd).  I'm not sure why that's failing.
I will look at them during this weekend, I already had a look at build 
log from

the phone, but it's better to look from a PC.


I had a closer look.  The failing tests were in python2 only, coming 
from the non-ascii (Gërman http://Königsgäßchen.de/straße and Japanese 
http://ヒ:キ@ヒ.abc.ニ/ヒ?キ#ワ) unicode url tests. So from one perspective we 
don't need to worry so much about them, we could just disable them (e.g. 
prepend @onlyPy3 to test_parse_url and test_url_vulnerabilities in 
test_util.py). We'll be dropping python2 any way in the near future.


On the other hand, given the nature of the vulnerabilities and the 
possible uses of urllib3, it's probably best not to leave python2 
untested, especially since they are known to pass on python2 anyway in 
the right conditions.  Probably there is some package that should be 
added to Build-Depends to enable python2 tests to pass, though I have no 
idea which that package might be.


Drew



Re: python-urllib3 1.25.6 uploaded to experimental (closes CVE-2019-11236) but fails build tests

2019-10-12 Thread Daniele Tricoli
Hi Drew,

On 12/10/2019 07:57, Drew Parsons wrote:
> Hi Daniele, just letting you know I uploaded python-urllib3 1.25.6 to
> experimental.

Thanks for taking care of urllib3.

> I was having some SSL trouble connecting to https://pub.orcid.org.  The error
> trace cited urllib3/contrib/pyopenssl.py, so I downloaded and installed
> python-urllib3 1.25.6 to see if updates to default SSL/TLS versions made any
> difference.  It didn't fix my problem, but since I had the package update 
> ready
> I figured I might as well present it to experimental.

I hope to have the time to investigate also this: urllib3/contrib/pyopenssl.py
contains code to have SSL with SNI_-support for Python 2 and it depends on
pyOpenSSL, cryptography and idna. Maybe looking at them can give us more clues.

Also, could you see if using Python3 the connection to https://pub.orcid.org 
work?

> The new version fixes CVE-2019-11236 (Bug#927172).  As far as I can tell it
> also fixes CVE-2019-11324 (Bug#927412), but I figured it's best to let you
> review that.
> 
> Th package build was successful on my system but gives build-time errors in
> chroot (on buildd).  I'm not sure why that's failing.

I will look at them during this weekend, I already had a look at build log from
the phone, but it's better to look from a PC.

Regards,

-- 
  Daniele Tricoli 'eriol'
  https://mornie.org



signature.asc
Description: OpenPGP digital signature