[issue19996] httplib infinite read on invalid header

2015-01-25 Thread Roundup Robot

Roundup Robot added the comment:

New changeset 25ecf3d0ea03 by Benjamin Peterson in branch '3.4':
handle headers with no key (closes #19996)
https://hg.python.org/cpython/rev/25ecf3d0ea03

New changeset 29923a9987be by Benjamin Peterson in branch 'default':
merge 3.4 (#19996)
https://hg.python.org/cpython/rev/29923a9987be

New changeset 9a4882b12218 by Benjamin Peterson in branch '2.7':
simply ignore headers with no name (#19996)
https://hg.python.org/cpython/rev/9a4882b12218

--
nosy: +python-dev
resolution:  - fixed
stage:  - resolved
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19996] httplib infinite read on invalid header

2014-08-02 Thread Jason Robinson

Jason Robinson added the comment:

I took the patches and verified that;

* running the new tests without the changed code in Lib/email/feedparser.py 
(head) and Lib/httplib.py, Lib/rfc822.py (2.7) fails both the new tests.
* running the new tests with the changed code passes the tests (on both head 
and 2.7).

Sainsburys Bank site seems to have been fixed thus verification in the first 
comment does not work - but the test I think emulates that well enough.

--
nosy: +ezio.melotti, jaywink
versions: +Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19996] httplib infinite read on invalid header

2014-06-27 Thread brian yardy

brian yardy added the comment:

import http.client
h = http.client.HTTPConnection('http://www.einstantloan.co.uk/')
h.request('GET', '/', headers={'Accept-Encoding': 'gzip'})
r = h.getresponse()
hdrs = r.getheaders()
body = r.read()  # Hang here.

curl --compressed http://www.einstantloan.co.uk/

--
nosy: +brianyardy

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19996] httplib infinite read on invalid header

2014-01-11 Thread Cory Benfield

Cory Benfield added the comment:

Is there anything I can do to help move this forward? I appreciate you're all 
busy so I'm happy for this to take as long as it takes, I just wanted to make 
sure it's not blocked behind me.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19996] httplib infinite read on invalid header

2013-12-18 Thread Cory Benfield

Cory Benfield added the comment:

Ok, here's a patch for 2.7 as well.

I decided to allow the empty header names in rfc822.py as well, if only because 
I wanted the changed parsing code to match. If anyone thinks that's an 
excessive change, I'll happily remove it.

--
Added file: http://bugs.python.org/file33198/hdrs_27.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19996] httplib infinite read on invalid header

2013-12-16 Thread Cory Benfield

Changes by Cory Benfield c...@lukasa.co.uk:


--
components: Library (Lib)
nosy: Lukasa
priority: normal
severity: normal
status: open
title: httplib infinite read on invalid header
type: behavior
versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19996] httplib infinite read on invalid header

2013-12-16 Thread Cory Benfield

New submission from Cory Benfield:

Initially spotted on Requests GitHub bugtracker: 
https://github.com/kennethreitz/requests/issues/1804

On receiving an HTTP response with an invalid header, httplib stops parsing the 
headers and attempts to receive the rest of the message as body content. 
Normally that would be fine, but problems occur if later on in the headers 
Transfer-Encoding: chunked is declared. This leads to a hang while reading 
the body content until the remote end forcibly closes the connection.

This bug certainly affects versions 2.7 through 3.3.

To reproduce (note that we need to request gzip to get the server to send the 
bad header):

import http.client
h = http.client.HTTPConnection('www.sainsburysbank.co.uk')
h.request('GET', '/', headers={'Accept-Encoding': 'gzip'})
r = h.getresponse()
hdrs = r.getheaders()
body = r.read()  # Hang here.

cURL configured equivalently doesn't exhibit this problem, that is the 
following works fine:

curl --compressed http://www.sainsburysbank.co.uk/


It's not clear to me that this behaviour is wrong. The server is definitely 
violating RFC 2616 which expressly forbids empty header names. I'm open to 
consultation about what the correct fix should be here (which may be nothing at 
all).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19996] httplib infinite read on invalid header

2013-12-16 Thread R. David Murray

R. David Murray added the comment:

Well, having it hang forever is a potential DOS attack, so something needs to 
be fixed, I think.

--
nosy: +christian.heimes, r.david.murray

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19996] httplib infinite read on invalid header

2013-12-16 Thread Cory Benfield

Cory Benfield added the comment:

The easiest way to 'fix' the DoS problem is to throw an exception if an invalid 
header is parsed. That's a backwards-compatibility problem though: things that 
previously 'worked' now won't. That presumably limits the ability to back-apply 
this fix to 2.7.7.

An alternative option is to speculatively attempt to parse the next line for 
headers or the end of the header block. I'm not sure this is a great idea: at 
this stage all we know is that the header block is malformed, so it's not clear 
that 'doing our best' is a good idea either, especially since that attitude got 
us here to begin with.

The best 'middle of the road' option is to abort message parsing at this stage 
without throwing an exception. This leads to truncated headers and no body, 
where previously we'd have got truncated headers and a body that potentially 
included the missing headers. We could also potentially add a warning about the 
problem.

Are there any preferences for a fix here, or a better solution than the above 
(none of which I'm wild about)?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19996] httplib infinite read on invalid header

2013-12-16 Thread R. David Murray

R. David Murray added the comment:

I haven't looked at the code, but could we preserve the existing behavior but 
apply a timeout to mitigate the DOS?

On the other hand, the fact that curl manages to return something indicates 
there is probably an error recovery strategy that would work.  I'm not sure if 
we have an error reporting mechanism in httplib if we do error recovery.  We do 
in the email module, and httplib uses the email code for headers, I think, so 
there might be a way to leverage that if there is no existing mechanism.  But 
of course even deciding to do that requires some discussion :)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19996] httplib infinite read on invalid header

2013-12-16 Thread Arfrever Frehtes Taifersar Arahesis

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:


--
nosy: +Arfrever

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19996] httplib infinite read on invalid header

2013-12-16 Thread Cory Benfield

Cory Benfield added the comment:

Maybe. If we do it we have to apply that timeout to all the socket actions on 
that HTTP connection. This would have the effect of changing the default value 
of the timeout parameter on the HTTPConnection object from 
socket._GLOBAL_DEFAULT_TIMEOUT to whatever value was chosen. We could do this 
for reads only, and avoid applying the timeout to connect() calls, but that's 
kind of weird.

We hit the same problem though: by default, HTTPConnections block indefinitely 
on all socket calls: we'd be changing that default to some finite timeout 
instead. Does that sound like a good way to go?

As for curl's error recovery strategy, I'm pretty sure it just keeps parsing 
the header block. That can definitely be done here. We do have an error 
reporting mechanism as well (sort of): we set the HTTPMessage.status field to 
some error string. We could do that, and continue to parse the header block: 
that's probably the least destructive way to fix this.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19996] httplib infinite read on invalid header

2013-12-16 Thread Cory Benfield

Cory Benfield added the comment:

An update: in Python 2.7 through 3.3, fixing this should only affect 
http.client/httplib, because they do most of their header parsing themselves. 
Fixing this in later versions of Python is more interesting, as http.client got 
rewritten to use email.parser (which uses email.feedparser). This means any 
change to fix this problem in HTTP will also affect anything else that uses 
this module. Not sure how problematic that is yet.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19996] httplib infinite read on invalid header

2013-12-16 Thread Cory Benfield

Cory Benfield added the comment:

Actually, that might be OK. I don't know the email package at all, but I 
suspect being able to handle empty header keys (by ignoring them) is a 
reasonable thing to do in the email case as well. Thoughts?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19996] httplib infinite read on invalid header

2013-12-16 Thread Cory Benfield

Cory Benfield added the comment:

Alright, here's a patch for the current tip. I'll need to prepare a different 
patch for earlier versions of Python, which will take me a little while longer 
to do (maybe not today). I've also signed a contributor agreement, but it 
doesn't look like that's propagated here yet.

--
keywords: +patch
Added file: http://bugs.python.org/file33169/hdrs.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19996] httplib infinite read on invalid header

2013-12-16 Thread R. David Murray

R. David Murray added the comment:

Heh.  A missing header *name* was something I never considered in the email 
package tests.  So yeah, that's worth handing one way or another.  I'll put 
reviewing this on my TODO list, since I'm the maintainer of the email package.

I'm updating the versions per our usual practice: the versions in which we want 
to fix it.

--
components: +email
nosy: +barry
versions: +Python 3.4 -Python 3.1, Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19996
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com