#19914: MemcachedCacheTests failing on pylibmc
-------------------------------------+-------------------------------------
Reporter: bpeschier | Owner: nobody
Type: Bug | Status: closed
Component: Core (Cache system) | Version: master
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by edmorley):
So I've dug into this a bit more, there are actually two tests that are
broken under pylibmc, and the above only covers the first.
== Issues with test_invalid_keys:
This test tries to set two different types of invalid keys:
1. a key that contains space characters
2. another that exceeds the max allowed length
Both python-memcached and pylibmc handle (2) fine. However (1) is
problematic with pylibmc, since it causes subsequent requests to the
server (ie teardown and later tests in the suite) to fail for the next few
seconds.
The cause of this is roughly:
* python-memcached validates key names before sending, whereas by
pylibmc doesn't by default (or more precisely, libmemcached doesn't by
default)
* therefore for pylibmc the invalid key name gets sent to the server,
rather than being rejected client-side
* this appears to expose some memcached server/libmemcache bug that
isn't pylibmc's fault (though I'm hoping for clarification, see:
https://github.com/lericson/pylibmc/issues/114#issuecomment-242929115)
Possible workarounds are:
a. not set keys names containing spaces (ie skip the test)
b. enable pylibmc's client side key validation passing `verify_keys`
within `behaviors` (just for this specific test)
c. use the binary protocol (since it's not affected)
In addition, since the bug only affects half of `test_invalid_keys()`, we
could split the test in two, to reduce the amount we skip/test in a non-
standard configuration.
Note: The issue presents itself in two forms depending on
libmemcached/memcached server versions. On Ubuntu 14.04 an assertion
occurs in libmemcached, which causes the whole test run to abort (example
in comment 2 above). Whereas on Ubuntu 16.04 a handful of exceptions are
seen and the rest of the tests complete fine. I'm pretty sure the
assertion *is* something pylibmc should be handling better, for which I've
filed:
https://github.com/lericson/pylibmc/issues/218
(though even with that fixed, the underlying ~server bug still needs to be
worked around)
== Issues with test_memcached_deletes_key_on_failed_set:
With the problematic test above skipped, the one remaining failure under
pylibmc is:
{{{
ERROR: test_memcached_deletes_key_on_failed_set
(cache.tests.MemcachedCacheTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/vagrant/src/_todo/django/tests/cache/tests.py", line 1233,
in test_memcached_deletes_key_on_failed_set
cache.set('small_value', large_value)
File
"/home/vagrant/src/_todo/django/django/core/cache/backends/memcached.py",
line 83, in set
if not self._cache.set(key, value, self.get_backend_timeout(timeout)):
Error: error 37 from memcached_set: SUCCESS
}}}
This is due to difference in behaviour between clients when the *value* is
too long.
Compare:
* When the *key* is too long:
- python-memcached: `memcache.MemcachedKeyLengthError: Key length is >
250`
- pylibmc: `ValueError: key length 251 too long, max is 250`
* When the *value* is too long:
- python-memcached: Returns successfully but didn't set the key-value
(this is [https://github.com/linsomniac/python-memcached/issues/32 by
design])
- pylibmc 1.5.1: `pylibmc.Error: error 37 from memcached_set: SUCCESS`
- pylibmc master: `pylibmc.TooBig`
Possible options:
a. just add a try-except around the `.set()` for both python-memcached
and pylibmc
b. differentiate between the two, and only expect exceptions for the
pylibmc case (either by just catching them, or by enforcing using
`assertRaises`)
--
Ticket URL: <https://code.djangoproject.com/ticket/19914#comment:4>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/067.d68c4a5fcc9b9c0f576944357347fc0c%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
