#16116: Compress cache data
-------------------------------------+-------------------------------------
Reporter: bjourne | Owner: nobody
Type: New feature | Status: closed
Component: Core (Cache system) | Version: 1.3
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
Has patch: 0 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 1 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by lucmult):
* cc: lucmult (added)
* ui_ux: => 0
Comment:
Hello guys,
Last week we run on this problem of memcache limitation and started to
have much more CPU load on our server due our biggest pages weren't being
cached.
python-memcached has support to zip (when zlib is available):
http://bazaar.launchpad.net/~python-memcached-team/python-
memcached/trunk/view/head:/memcache.py#L763
I'm willing to submit a patch to `MemcachedCache` backend to be able to
configure it on settings.
settings:
{{{#!python
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': ['127.0.0.1:11211'],
'OPTIONS': {
'CLIENT_OPTIONS': {'server_max_value_length': 67108864,
'pickleProtocol': 2},
'MIN_COMPRESS_LEN': 500,
}
}
}
}}}
`server_max_value_length` is useful because python-memcached uses it to
decide if it should ignore the value. It's default to `1024*1024`
http://bazaar.launchpad.net/~python-memcached-team/python-
memcached/trunk/view/head:/memcache.py#L775
In the docs I would suggest our users to set with the value got from
memcache stats:
{{{#!bash
$ echo "stats settings" | nc localhost 11211`
STAT maxbytes 67108864
...
}}}
Although this hasn't shown as the real limit on my local test :(
If the application send a value bigger than memcache can handle it just
respond as data is too big to store, no exception is raised, cache is only
ignored (as we have currently).
Our usage of `MIN_COMPRESS_LEN` has shown a significant decrease of
network traffic between django server and memcached server.
This fix would benefit more 2 tickets:
Memcached session "forgets" big values -
https://code.djangoproject.com/ticket/16358
Memcached using pickle highest protocol -
https://code.djangoproject.com/ticket/16378
--
Ticket URL: <https://code.djangoproject.com/ticket/16116#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
