#34681: Optimize memcache_key_warnings()
------------------------------------------------+--------------------------
Reporter: Adam Johnson | Owner: nobody
Type: Cleanup/optimization | Status: assigned
Component: Core (Cache system) | Version: dev
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
------------------------------------------------+--------------------------
The cache functino memcache_key_warnings() iterates the key character-by-
character, a slow operation in Python because it has to create and destroy
many individual `str` objects. Instead, we can search the string with a
regular expression for a 6x speedup.
IPython session comparing old and new approaches:
{{{
In [1]: import re
...:
...: MEMCACHE_MAX_KEY_LENGTH = 250
...:
...:
...: def old(key):
...: if len(key) > MEMCACHE_MAX_KEY_LENGTH:
...: yield (
...: "Cache key will cause errors if used with memcached:
%r "
...: "(longer than %s)" % (key, MEMCACHE_MAX_KEY_LENGTH)
...: )
...: for char in key:
...: if ord(char) < 33 or ord(char) == 127:
...: yield (
...: "Cache key contains characters that will cause
errors if "
...: "used with memcached: %r" % key
...: )
...: break
...:
...: memcached_error_chars_re = re.compile(r"[\x00-\x32\x127]")
...:
...:
...: def new(key):
...: if len(key) > MEMCACHE_MAX_KEY_LENGTH:
...: yield (
...: "Cache key will cause errors if used with memcached:
%r "
...: "(longer than %s)" % (key, MEMCACHE_MAX_KEY_LENGTH)
...: )
...: if memcached_error_chars_re.match(key):
...: yield (
...: "Cache key contains characters that will cause errors
if "
...: "used with memcached: %r" % key
...: )
...:
In [2]: %timeit list(old('acme-bookstore-user-1234567-book-78910391'))
1.35 µs ± 1.09 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops
each)
In [3]: %timeit list(new('acme-bookstore-user-1234567-book-78910391'))
212 ns ± 1.42 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops
each)
In [4]: %timeit list(old('homepage\n'))
545 ns ± 1.17 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops
each)
In [5]: %timeit list(new('homepage\n'))
209 ns ± 1.03 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops
each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34681>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/01070188fba768d8-e85a5275-9735-417d-889e-1ba6f839adfe-000000%40eu-central-1.amazonses.com.
