[issue21463] RuntimeError when URLopener.ftpcache is full
Skyler Leigh Amador added the comment: You're welcome, happy to help :) On Mon, Jun 9, 2014 at 8:31 AM, Erik Bray rep...@bugs.python.org wrote: Erik Bray added the comment: Thanks Skyler for finishing the job. I got busy/distracted. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21463 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21463 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21463] RuntimeError when URLopener.ftpcache is full
Erik Bray added the comment: Thanks Skyler for finishing the job. I got busy/distracted. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21463 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21463] RuntimeError when URLopener.ftpcache is full
Skyler Leigh Amador added the comment:
I've made a test for this patch with a very minimal mock ftpwrapper. We can see
it fails on dictionary size change without Erik's fix:
==
ERROR: test_ftp_cache_pruning (test.test_urllib.urlopen_HttpTests)
--
Traceback (most recent call last):
File /home/skyler/cpython/Lib/test/test_urllib.py, line 336, in
test_ftp_cache_pruning
urlopen('ftp://localhost')
File /home/skyler/cpython/Lib/test/test_urllib.py, line 45, in urlopen
return opener.open(url)
File /home/skyler/cpython/Lib/urllib/request.py, line 1631, in open
return getattr(self, name)(url)
File /home/skyler/cpython/Lib/urllib/request.py, line 1914, in open_ftp
for k in self.ftpcache.keys():
RuntimeError: dictionary changed size during iteration
--
nosy: +shiinee
Added file: http://bugs.python.org/file35522/urllib-request-ftpcache-test.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21463
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21463] RuntimeError when URLopener.ftpcache is full
Benjamin Peterson added the comment: (I left some comments on Rietveld.) -- nosy: +benjamin.peterson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21463 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21463] RuntimeError when URLopener.ftpcache is full
Skyler Leigh Amador added the comment: Addressed review comments -- Added file: http://bugs.python.org/file35529/urllib-request-ftpcache-test.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21463 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21463] RuntimeError when URLopener.ftpcache is full
Roundup Robot added the comment: New changeset b8f9ae84d211 by Benjamin Peterson in branch '3.4': in ftp cache pruning, avoid changing the size of a dict while iterating over it (closes #21463) http://hg.python.org/cpython/rev/b8f9ae84d211 New changeset 6f70a18313e5 by Benjamin Peterson in branch 'default': merge 3.4 (#21463) http://hg.python.org/cpython/rev/6f70a18313e5 -- nosy: +python-dev resolution: - fixed stage: test needed - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21463 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21463] RuntimeError when URLopener.ftpcache is full
Jessica McKellar added the comment:
I want to state explicitly what the error is for some new contributors who
might pick this up at a sprint this weekend:
The issue is that you can't change a dictionary while iterating over it:
d = {a: b}
for elt in d.keys():
... del d[elt]
...
Traceback (most recent call last):
File stdin, line 1, in module
RuntimeError: dictionary changed size during iteration
In Python 2, d.keys() produced a copy of the list of keys, and we delete items
from the dictionary while iterating over that copy, which is safe. In Python 3,
d.keys() produces a view object* instead, and mutating the dictionary while
iterating over it is unsafe and raises an error.
The patch makes a copy of the keys before iterating over it so that it is safe
in Python 3.
* https://docs.python.org/3/library/stdtypes.html#dict-views
--
nosy: +jesstess
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21463
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21463] RuntimeError when URLopener.ftpcache is full
New submission from Erik Bray: This is probably a pretty rare corner case, but a coworker reported this to me while testing code that does open several ftp connections to different files. -- components: Library (Lib) files: urllib-request-ftpcache-error.patch keywords: patch messages: 218170 nosy: erik.bray priority: normal severity: normal status: open title: RuntimeError when URLopener.ftpcache is full type: crash Added file: http://bugs.python.org/file35199/urllib-request-ftpcache-error.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21463 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21463] RuntimeError when URLopener.ftpcache is full
Changes by Berker Peksag berker.pek...@gmail.com: -- nosy: +orsenthil ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21463 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21463] RuntimeError when URLopener.ftpcache is full
Éric Araujo added the comment: Thanks for the report and patch. Would you mind adding a unit test? (Note that the “crash” type is for segfaults, not Python exceptions.) -- nosy: +eric.araujo stage: - test needed type: crash - behavior versions: +Python 2.7, Python 3.4, Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21463 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21463] RuntimeError when URLopener.ftpcache is full
Erik Bray added the comment: Ah, didn't know that about crash. I wanted to add a test but hesitated only because that code is not well tested to begin with (perhaps, hence, this going unnoticed for so long). But I guess it could be done by mocking the ftpwrapper class. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21463 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
