Hi everyone, Is there a valid reason why locmem is pickling data? >From what I understand, locmem is doing that just because its behaviour was copied from other caches. But since locmem is basically just a dictionary stored in the current Python process without persistence, there is no need to serialize data. By skipping the pickling step, we avoid creating a new object and we avoid the CPU time needed for this serialization. So that’s better in terms of memory and CPU.
I created a gist containing the unpickling version of LocMemCache: https://gist.github.com/BertrandBordage/7f33b3ccf2686e8da9ed I tested it on the test suite of django-cachalot, everything works fine including thread-safe tests. It also scores drastically better when caching medium/large objects: Using the current locmem backend: In [1]: %timeit cache.set('something', list(range(10000))) 1000 loops, best of 3: 429 µs per loop In [2]: %timeit cache.get('something') 1000 loops, best of 3: 377 µs per loop Using the unpickling version of locmem: In [1]: %timeit cache.set('something', list(range(10000))) 10000 loops, best of 3: 179 µs per loop In [2]: %timeit cache.get('something') 10000 loops, best of 3: 23.8 µs per loop Regards, Bertrand -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a45ce426-38b4-4543-8824-2e085e6acd83%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
