#20584: Django's Memcached backend get_many() doesn't handle iterators
-------------------------------------+-------------------------------------
Reporter: guyon.moree@… | Owner: Christian
| Barcenas
Type: Bug | Status: assigned
Component: Core (Cache system) | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by Christian Barcenas:
Old description:
> When the `keys` parameter to `get_many()` is an iterator, its values will
> be consumed in a list comprehension, but then later the already-consumed
> iterator is passed to `zip`.
>
> https://github.com/django/django/blob/master/django/core/cache/backends/memcached.py#L90
>
> This causes a very confusing `ValueError` which is raised when the cache
> backend attempts to map lower-level Memcache-backend cache keys back to
> higher-level Djanco cache keys.
>
> {{{
> Traceback (most recent call last):
> File "[snip]/django/tests/cache/tests.py", line 1345, in
> test_get_many_accepts_iterator
> values = cache.get_many(iter(['fizz']))
> File "[snip]/django/lib/python3.6/site-
> packages/Django-2.2.dev20180706090656-py3.6.egg/django/core/cache/backends/memcached.py",
> line 91, in get_many
> return {m[k]: v for k, v in ret.items()}
> File "[snip]/django/lib/python3.6/site-
> packages/Django-2.2.dev20180706090656-py3.6.egg/django/core/cache/backends/memcached.py",
> line 91, in <dictcomp>
> return {m[k]: v for k, v in ret.items()}
> KeyError: ':1:fizz'
> }}}
New description:
When the `keys` parameter to `get_many()` is an iterator, its values will
be consumed in a list comprehension. However, later the already-consumed
iterator is then passed to `zip`.
https://github.com/django/django/blob/master/django/core/cache/backends/memcached.py#L90
This causes a very confusing `KeyError` which is raised when the cache
backend attempts to map lower-level Memcache-backend cache keys back to
higher-level Djanco cache keys.
{{{
Traceback (most recent call last):
File "[snip]/django/tests/cache/tests.py", line 1345, in
test_get_many_accepts_iterator
values = cache.get_many(iter(['fizz']))
File "[snip]/django/lib/python3.6/site-
packages/Django-2.2.dev20180706090656-py3.6.egg/django/core/cache/backends/memcached.py",
line 91, in get_many
return {m[k]: v for k, v in ret.items()}
File "[snip]/django/lib/python3.6/site-
packages/Django-2.2.dev20180706090656-py3.6.egg/django/core/cache/backends/memcached.py",
line 91, in <dictcomp>
return {m[k]: v for k, v in ret.items()}
KeyError: ':1:fizz'
}}}
--
--
Ticket URL: <https://code.djangoproject.com/ticket/20584#comment:6>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/079.602d35075fa1a780c5c14282a672e8ba%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
