New submission from Fabian:
While testing pywikibot using requests and urllib3 on Python 3.6 we got an
interesting error:
==
ERROR: testQueryApiGetter (tests.wikidataquery_tests.TestApiSlowFunctions)
Test that we can actually retreive data and that caching works.
--
Traceback (most recent call last):
File /home/travis/build/xZise/pywikibot-core/tests/wikidataquery_tests.py,
line 252, in testQueryApiGetter
data = w.query(q)
File
/home/travis/build/xZise/pywikibot-core/pywikibot/data/wikidataquery.py, line
601, in query
data = self.getDataFromHost(fullQueryString)
File
/home/travis/build/xZise/pywikibot-core/pywikibot/data/wikidataquery.py, line
563, in getDataFromHost
resp = http.fetch(url)
File /home/travis/build/xZise/pywikibot-core/pywikibot/comms/http.py, line
359, in fetch
error_handling_callback(request)
File /home/travis/build/xZise/pywikibot-core/pywikibot/comms/http.py, line
276, in error_handling_callback
raise request.data
File /home/travis/build/xZise/pywikibot-core/pywikibot/comms/http.py, line
255, in _http_process
auth=auth, timeout=timeout, verify=True)
File
/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/sessions.py,
line 465, in request
resp = self.send(prep, **send_kwargs)
File
/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/sessions.py,
line 573, in send
r = adapter.send(request, **kwargs)
File
/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/adapters.py,
line 337, in send
conn = self.get_connection(request.url, proxies)
File
/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/adapters.py,
line 251, in get_connection
conn = self.poolmanager.connection_from_url(url)
File
/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/packages/urllib3/poolmanager.py,
line 139, in connection_from_url
return self.connection_from_host(u.host, port=u.port, scheme=u.scheme)
File
/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/packages/urllib3/poolmanager.py,
line 125, in connection_from_host
self.pools[pool_key] = pool
File
/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/packages/urllib3/_collections.py,
line 66, in __setitem__
_key, evicted_value = self._container.popitem(last=False)
KeyError: ('https', 'eu.wiktionary.org', 443)
Now that doesn't make much sense, as OrderedDict.popitem() only returns a
KeyError if it's empty but then not with that error message. Unfortunately I
can't reproduce it without doing the complete pywikibot test suite. When I only
execute the tests where it occurred I don't get any failures. So I modified the
output in urllib3 and returned the key as well as the content before
popitem(last=False) is called (I shortened the values):
Key: ('https', 'bn.wikipedia.org', 443)
Content: OrderedDict([(('https', 'bn.wikipedia.org', 443),
requests.packages.urllib3.connectionpool.HTTPSConnectionPool object at
0x7fe483d991d0), (('https', 'bs.wikipedia.org', 443), …0x7fe483d99470),
(('https', 'ca.wikipedia.org', 443), …0x7fe483da0e10), (('https',
'cs.wikipedia.org', 443), …0x7fe483dfc908), (('https', 'da.wikipedia.org',
443), …0x7fe483d98c88), (('https', 'de.wikipedia.org', 443),
…0x7fe4911e06a0), (('https', 'diq.wikipedia.org', 443), …0x7fe483d6b400),
(('https', 'dsb.wikipedia.org', 443), …0x7fe483d6bcf8), (('https',
'en.wikipedia.org', 443), …0x7fe483dc1da0), (('https', 'eo.wikipedia.org',
443), …0x7fe483d6cf28), (('https', 'es.wikipedia.org', 443),
…0x7fe483d6b4a8), (('https', 'fa.wikipedia.org', 443), …0x7fe483d27278),
(('https', 'fi.wikipedia.org', 443), …0x7fe483d6ce10), (('https',
'fr.wikipedia.org', 443), …0x7fe483e24cf8), (('https', 'frr.wikipedia.org',
443), …0x7fe483d39390), (('https', 'ga.wikipedia.org', 443),
…0x7fe483c99668), (('https', 'gl.wikipedia.org', 443), …0x7fe483ca3f98),
(('https', 'als.wikipedia.org', 443), …0x7fe483c4fcf8), (('https',
'hu.wikipedia.org', 443), …0x7fe483cd09e8)])
As you can see it is not empty and the key in the KeyError is the first key in
the OrderedDict. Also the key there is different from the key in the original
exception I noticed so it's not a specific key that failed.
I don't think versions before Python 3.6 are affected as we had tests running
on Python 3.5 (before Travis switched to 3.6 recently) and these all worked.
Also not all popitem() calls in that line fail. I'm using Python 3.6.0a0
(default:d6c91b8242d2, Jul 18 2015, 16:36:01).
See also: https://github.com/shazow/urllib3/issues/680 and
https://phabricator.wikimedia.org/T106212
--
components: Library (Lib)
messages: 246937
nosy: xZise
priority: normal
severity: normal
status: open
title: OrderedDict.popitem() raises KeyError
