New submission from Ned Deily <n...@acm.org>: [NOTE: applies to 2.x urllib2 and similar code in merged 3.x urllib]
test_urllib2 can fail because urllib2.FileHandler assumes incorrectly that the local host has only a single IP address. It is not uncommon to have host IP configurations where a host has more than one network interface and the same IP host name is associated with each address. Both the urllib module and test_urllib2 use socket.gethostbyname(socket.gethostname()) to find "the" host IP address. But, as can be seen here, consecutive calls may produce different addresses depending on the network configuration and underlying os implementation: Python 2.6.1 (r261:67515, Dec 17 2008, 23:27:50) [GCC 4.0.1 (Apple Inc. build 5490)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> socket.gethostbyname(socket.gethostname()) '10.52.12.105' >>> socket.gethostbyname(socket.gethostname()) '10.52.12.105' >>> socket.gethostbyname(socket.gethostname()) '10.52.12.205' >>> This leads to predictable test failures when the calls in test_urllib2 and urllib2.FileHandler return different addresses: test_urllib2 test test_urllib2 failed -- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/test/te st_urllib2.py", line 621, in test_file r = h.file_open(Request(url)) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2 .py", line 1229, in file_open return self.open_local_file(req) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2 .py", line 1266, in open_local_file raise URLError('file not on local host') URLError: <urlopen error file not on local host> The simplest way to avoid the test failure is to modify urllib2.FileHandler to use socket.gethostbyname_ex which returns all of the IPv4 addresses associated with a hostname: >>> socket.gethostbyname_ex(socket.gethostname()) ('myhost.net', [], ['10.52.12.205', '10.52.12.105']) Attached patches for 2.x urllib2 and 3.x urllib do that. Note that there remain other issues in this area: - when urllib2 is enhanced to support IPv6, code is needed to return all of the host's IPv6 addresses as well (-> adding a note to open Issue1675455) - the merged 3.0 urlib has two nearly identical functions named open_local_file, one each from 2.x urllib.URLopener and urllib2.FileHandler, and both use similarly flawed socket.gethostbyname(socket.gethostname()) tests but the tests for local vs remote file URLs is somewhat different in each. (The patches here do not attempt to address this other than to add a comment.) ---------- components: Library (Lib) files: patch-nad0017-trunk-26.txt messages: 84806 nosy: nad severity: normal status: open title: test_urllib2 fails - urlopen error file not on local host versions: Python 2.6, Python 2.7, Python 3.0, Python 3.1 Added file: http://bugs.python.org/file13514/patch-nad0017-trunk-26.txt _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue5625> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com