[issue14044] IncompleteRead error with urllib2 or urllib.request -- fine with urllib, wget, or curl

[CJ Kucera]

CJ Kucera added the comment:

Ah, well, actually I suppose I'll rescind that a bit - other pages about this 
bug around the internet had been claiming that the 'requests' module uses 
urllib in the backend and was subject to this bug as well, but after 
experimenting myself, it seems like if that IS the case, they're working around 
it somehow, because using requests makes this succeed 100% of the time.  I 
probably should've tried that first!

So anyway, there's a reasonable workaround, at least.  Sorry for the bugspam!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14044] IncompleteRead error with urllib2 or urllib.request -- fine with urllib, wget, or curl

[CJ Kucera]

CJ Kucera added the comment:

I've just encountered this problem on Python 3.6, on a different URL.  The 
difference being that it's not encountered with EVERY page load, though I'd say 
it happens with at least half:

import urllib.request
html = urllib.request.urlopen('http://www.basicinstructions.net/').read()
print('Succeeded!')

I realize that the root problem here may be an HTTP server doing something 
improper, but I've got no way of fixing someone else's webserver.  It'd be 
really nice if there was a reasonable way of handling this in Python itself.  
As mentioned in the original report, other methods of retreiving this URL work 
without fail (curl/wget/etc).  As it is, the only way for me to be sure of 
retreiving the entire page contents is by looping until I don't get an 
IncompleteRead, which is hardly ideal.

--
nosy: +apocalyptech
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14044] IncompleteRead error with urllib2 or urllib.request -- fine with urllib, wget, or curl

[Martin Panter]

Martin Panter added the comment:

Closing this as being a bug in the web server, rather than Python.

If someone wants to add a way to force a HTTP 1.0 response, or a way to get all 
valid data before raising the exception, I suggest opening a new report.

--
resolution:  -> third party
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14044] IncompleteRead error with urllib2 or urllib.request -- fine with urllib, wget, or curl

[Demian Brecht]

Changes by Demian Brecht demianbre...@gmail.com:


--
nosy:  -demian.brecht

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14044
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14044] IncompleteRead error with urllib2 or urllib.request -- fine with urllib, wget, or curl

[Laurento Frittella]

Laurento Frittella added the comment:

Even if forcing the HTTP/1.0 workaround works it can end up in weird issues, 
especially if used in something more than a small script, like the one I tried 
to describe in this issue report[1] for the requests python library.

[1] https://github.com/kennethreitz/requests/issues/2341

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14044
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14044] IncompleteRead error with urllib2 or urllib.request -- fine with urllib, wget, or curl

[Martin Panter]

Martin Panter added the comment:

I suggest this is the same situation as Issue 6785, and is not a bug in Python. 
However it might be reasonable to allow forcing a HTTP client connection to 
version 1.0, which could be used as a workaround.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14044
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14044] IncompleteRead error with urllib2 or urllib.request -- fine with urllib, wget, or curl

[Demian Brecht]

Changes by Demian Brecht demianbre...@gmail.com:


--
nosy: +demian.brecht

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14044
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14044] IncompleteRead error with urllib2 or urllib.request -- fine with urllib, wget, or curl

[Laurento Frittella]

Laurento Frittella added the comment:

I had the same problem using urllib2 and the following trick worked for me

import httplib
httplib.HTTPConnection._http_vsn = 10
httplib.HTTPConnection._http_vsn_str = 'HTTP/1.0'

Source: http://stackoverflow.com/a/20645845

--
nosy: +laurento.frittella

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14044
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14044] IncompleteRead error with urllib2 or urllib.request -- fine with urllib, wget, or curl

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


--
nosy: +serhiy.storchaka
type:  - behavior
versions: +Python 3.3, Python 3.4 -Python 2.6, Python 3.1, Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14044
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14044] IncompleteRead error with urllib2 or urllib.request -- fine with urllib, wget, or curl

[Mathieu Sornay]

Changes by Mathieu Sornay msor...@gmail.com:


--
nosy: +lechfeck

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14044
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14044] IncompleteRead error with urllib2 or urllib.request -- fine with urllib, wget, or curl

[raylu]

raylu added the comment:

The URL works for me.

While wget does download it successfully, I get the following output:

$ wget 
http://info.kingcounty.gov/health/ehs/foodsafety/inspections/XmlRest.aspx\?Zip_Code\=98199
--2013-06-13 12:15:21--  
http://info.kingcounty.gov/health/ehs/foodsafety/inspections/XmlRest.aspx?Zip_Code=98199
Resolving info.kingcounty.gov (info.kingcounty.gov)... 146.129.240.75
Connecting to info.kingcounty.gov (info.kingcounty.gov)|146.129.240.75|:80... 
connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/xml]
Saving to: ‘XmlRest.aspx?Zip_Code=98199’

[  =   ] 515,315  
448KB/s   in 1.1s   

2013-06-13 12:15:23 (448 KB/s) - Read error at byte 515315 (Success).Retrying.

--2013-06-13 12:15:24--  (try: 2)  
http://info.kingcounty.gov/health/ehs/foodsafety/inspections/XmlRest.aspx?Zip_Code=98199
Connecting to info.kingcounty.gov (info.kingcounty.gov)|146.129.240.75|:80... 
connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/xml]
Saving to: ‘XmlRest.aspx?Zip_Code=98199’

[ =] 0   
--.-K/s   in 0s  


Cannot write to ‘XmlRest.aspx?Zip_Code=98199’ (Success).

Similarly, curl gives

$ curl 
http://info.kingcounty.gov/health/ehs/foodsafety/inspections/XmlRest.aspx\?Zip_Code\=98199
  /dev/null
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
100  503k0  503k0 0   222k  0 --:--:--  0:00:02 --:--:--  229k
curl: (18) transfer closed with outstanding read data remaining

$ wget --version
GNU Wget 1.14 built on linux-gnu.

$ curl --version
curl 7.30.0 (x86_64-pc-linux-gnu) libcurl/7.30.0 OpenSSL/1.0.1e zlib/1.2.8 
libidn/1.25 libssh2/1.4.2 librtmp/2.3

--
nosy: +raylu

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14044
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14044] IncompleteRead error with urllib2 or urllib.request -- fine with urllib, wget, or curl

[Antoine Pitrou]

Antoine Pitrou added the comment:

The example URL doesn't seem to work anymore. Do you have another example to 
test with?

--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14044
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14044] IncompleteRead error with urllib2 or urllib.request -- fine with urllib, wget, or curl

[Alex Quinn]

New submission from Alex Quinn aq2...@alexquinn.org:

When accessing this URL, both urllib2 (Py2) and urlib.client (Py3) raise an 
IncompleteRead error.
http://info.kingcounty.gov/health/ehs/foodsafety/inspections/XmlRest.aspx?Zip_Code=98199

Previous discussions about similar errors suggest that this may be due to a 
problem with the server and chunked data transfer.  (See links below.)  I can't 
understand what that means.  However, this works fine with urllib (Py2), curl, 
wget, and all regular web browsers I've tried it with.  Thus, I would have 
expected urllib2 (Py2) and urllib.request (Py3) to cope with it similarly.

Versions I've tested with:
- Fails with urllib2 + Python 2.5.4, 2.6.1, 2.7.2  (Error messages vary.)
- Fails with urllib.request + Python 3.1.2, 3.2.2
- Succeeds with urllib + Python 2.5.4, 2.6.1, 2.7.2
- Succeeds with wget 1.11.1
- Succeeds with curl 7.15.5

___
TEST CASES

# FAILS - Python 2.7, 2.6, 2.5
import urllib2
url = 
http://info.kingcounty.gov/health/ehs/foodsafety/inspections/XmlRest.aspx?Zip_Code=98199;
xml_str = urllib2.urlopen(url).read() # Raises httplib.IncompleteRead

# FAILS - Python 3.2, 3.1
import urllib.request
url = 
http://info.kingcounty.gov/health/ehs/foodsafety/inspections/XmlRest.aspx?Zip_Code=98199;
xml_str = urllib.request.urlopen(url).read() # Raises http.client.IncompleteRead

# SUCCEEDS - Python 2.7, 2.6, 2.5
import urllib
url = 
http://info.kingcounty.gov/health/ehs/foodsafety/inspections/XmlRest.aspx?Zip_Code=98199;
xml_str = urllib.urlopen(url).read()
dom = xml.dom.minidom.parseString(xml_str) # Verify XML is complete
print(urllib:  %d bytes received and parsed successfully%len(xml_str))

# SUCCEEDS - wget
wget -O- 
http://info.kingcounty.gov/health/ehs/foodsafety/inspections/XmlRest.aspx?Zip_Code=98199;
 | wc

# SUCCEEDS - curl - prints an error, but returns the full data anyway
curl 
http://info.kingcounty.gov/health/ehs/foodsafety/inspections/XmlRest.aspx?Zip_Code=98199;
 | wc

___
RELATED DISCUSSIONS

http://www.gossamer-threads.com/lists/python/python/847985
http://bugs.python.org/issue11463  (closed)
http://bugs.python.org/issue6785   (closed)
http://bugs.python.org/issue6312   (closed)

--
components: Library (Lib)
messages: 153581
nosy: Alex Quinn
priority: normal
severity: normal
status: open
title: IncompleteRead error with urllib2 or urllib.request -- fine with urllib, 
wget, or curl
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14044
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14044] IncompleteRead error with urllib2 or urllib.request -- fine with urllib, wget, or curl

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
nosy: +orsenthil

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14044
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com