On Thu, Feb 18, 2010 at 2:21 PM, Forest Bond <for...@alittletooquiet.net> wrote: > Hi, > > On Thu, Feb 18, 2010 at 03:17:32PM -0500, Alex Gaynor wrote: >> On Thu, Feb 18, 2010 at 3:12 PM, Jeremy Dunck <jdu...@gmail.com> wrote: >> > On Thu, Feb 18, 2010 at 1:34 PM, ramu...@gmail.com <ramu...@gmail.com> >> > wrote: >> >> Does anyone know why django can't keep a data bigger, than 1 mb in >> >> memcached ? >> >> >> >> This is no big deal to cut big data to 1mb pieces before setting and >> >> merge this pieces after getting from memcached. And this algorithm can >> >> work transparently for users. >> > >> > I think Ramusus is asking for the low-level caching framework to split >> > serialized data into 1MB chunks before storing in a bunch of keys if >> > the data is larger than 1MB. >> > >> > He's not asking why there's a limit, he's asking for Django to work >> > around the limit. >> > >> > I think the general reasoning for a 1MB limit is that it places bounds >> > on access/storage times for a given key, and avoids an antipattern of >> > storing really large objects in cache. I've wished for special-case >> > side-stepping of the 1MB limit before myself, but I'd hate for it to >> > be the default. >> >> I don't see how that would be possible, sure storing it would be easy, >> but how do you retrieve it? Either you have to always try the >> "part-2" key (unreasonable overhead fro 99% of the use cases) or you >> have to store a tuple of (blocks, data) in the first slot, that's not >> a huge runtime overhead, but it feels like a large code-complexity >> overhead for a very rare usecase (I had no idea memcached even had >> these limits, and I use it pretty extensively).
Attempting to store >1MB fails silently, so you may have hit this but never noticed. Having a hot >1MB miss kind of sucks. :-) > Such operations would also not be atomic. Concurrent attempts to update cache > would be problematic. > To be clear, I'm not arguing that this should be included in Django, especially by default. I can roll my own when needed -- that's sort of the point of the low-level API. But atomic semantics can be achieved with prefixing: http://www.aminus.org/blogs/index.php/2007/12/30/memcached_set_invalidation?blog=2 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
