[issue15002] urllib2 does not download 4 MB file completely using ftp

Irit Katriel added the comment: 2.7 backport is no longer relevant. -- nosy: +iritkatriel status: open -> closed ___ Python tracker ___

[issue15002] urllib2 does not download 4 MB file completely using ftp

Roundup Robot added the comment: New changeset bb71b71322a3 by Senthil Kumaran in branch '3.4': urllib.response object to use _TemporaryFileWrapper (and _TemporaryFileCloser) http://hg.python.org/cpython/rev/bb71b71322a3 New changeset 72fe23edfec6 by Senthil Kumaran in branch '3.4': NEWS entry

[issue15002] urllib2 does not download 4 MB file completely using ftp

Senthil Kumaran added the comment: This is fixed in 3.4 and 3.5. I will backport to 2.7 ( I think, it is worth it). -- resolution: - fixed stage: patch review - resolved ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15002

[issue15002] urllib2 does not download 4 MB file completely using ftp

Senthil Kumaran added the comment: Christian's patch is good.It helps in setting the socket.makefile file descriptor to a well behaving well file close wrapper and thus will help us prevent some tricky fd close issues. I added tests for coverage to ensure that we are asserting the type and

[issue15002] urllib2 does not download 4 MB file completely using ftp

Christian Theune added the comment: Looking into this. It seems that it doesn't happen for all servers, I can download large files reliably from other sources. I'll make another wireshark recording to get more details for me to analyze. -- nosy: +ctheune

[issue15002] urllib2 does not download 4 MB file completely using ftp

Senthil Kumaran added the comment: I'll make another wireshark recording to get more details for me to analyze. Thank you! That will be useful. Please test it against 3.x version as it has seen cleanups recently. -- ___ Python tracker

[issue15002] urllib2 does not download 4 MB file completely using ftp

Christian Theune added the comment: This is actually the same problem as #18879. Changing the sample to keep a reference to the addinfourl object avoids this issue. This is even worse than #18879 in the sense that the error goes undetected and just leaves you with partial data. Looking at

[issue15002] urllib2 does not download 4 MB file completely using ftp

Changes by Christian Theune c...@gocept.com: -- versions: +Python 3.4, Python 3.5 -Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15002 ___

[issue15002] urllib2 does not download 4 MB file completely using ftp

Christian Theune added the comment: I wasn't able to come up with a good testcase. :( I tried similar approaches as in #18879 but I wasn't able to make them trigger the behaviour as it also seems to be an issue regarding actual network performance ... :/ Backport to 2.7 is currently missing

[issue15002] urllib2 does not download 4 MB file completely using ftp

Changes by Christian Theune c...@gocept.com: -- hgrepos: +237 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15002 ___ ___ Python-bugs-list mailing

[issue15002] urllib2 does not download 4 MB file completely using ftp

Changes by Thomas Wouters tho...@python.org: -- stage: needs patch - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15002 ___ ___

[issue15002] urllib2 does not download 4 MB file completely using ftp

Christian Theune added the comment: Antoine, I'm adding you here as I'm leveraging your patch from #18879. I'd need some feedback about the backport, but this patch should be OK for 3.4. Also, if you had an idea how to test this - I tried, but failed so far. --

[issue15002] urllib2 does not download 4 MB file completely using ftp

Senthil Kumaran added the comment: Christian , with respect to patch, I agree with the logic (using something similar to #18879). Does all current unittests succeed with this? (I suspect not) A unittest for coverage would be helpful. -- ___ Python

[issue15002] urllib2 does not download 4 MB file completely using ftp

Changes by Christian Theune c...@gocept.com: -- keywords: +patch Added file: http://bugs.python.org/file34915/d3c6ab639306.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15002 ___

[issue15002] urllib2 does not download 4 MB file completely using ftp

Antoine Pitrou added the comment: Well, this looks ok on the principle, but I haven't investigated the urllib-specific parts, so I'll let Senthil delve into this :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15002

[issue15002] urllib2 does not download 4 MB file completely using ftp

Christian Theune added the comment: Antoine, could you check my last comment in here? (The nosy list got reset accidentally when I made that comment and got a conflict from the tracker). -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org

[issue15002] urllib2 does not download 4 MB file completely using ftp

Brian Brazil brian.bra...@gmail.com added the comment: I've tested this on head, and the issue appears to be buggy ftp code in python. From the attached tcpdump for fetching delegated-ripencc-20120706: 12:57:19.933607 IP myhost.39627 ftp.ripe.net.ftp: Flags [.], ack 511, win 115, options

[issue15002] urllib2 does not download 4 MB file completely using ftp

Changes by Antoine Pitrou pit...@free.fr: -- nosy: +giampaolo.rodola ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15002 ___ ___ Python-bugs-list

[issue15002] urllib2 does not download 4 MB file completely using ftp

Brian Brazil brian.bra...@gmail.com added the comment: More particularly, the ftpwrapper's ftp member is being GCed sometime after FtpHandler.ftp_open returns. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15002

[issue15002] urllib2 does not download 4 MB file completely using ftp

Changes by sspapilin sspapi...@gmail.com: -- title: urllib2 does not download 4 MB file fully using ftp - urllib2 does not download 4 MB file completely using ftp ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15002

[issue15002] urllib2 does not download 4 MB file completely using ftp

R. David Murray rdmur...@bitdance.com added the comment: The same problem exists in Python3. -- nosy: +orsenthil, r.david.murray stage: - needs patch versions: +Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org

[issue15002] urllib2 does not download 4 MB file completely using ftp

Senthil Kumaran sent...@uthcode.com added the comment: That's surprising! I shall test it with http debug mode and see what's happening. -- assignee: - orsenthil ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15002