I'm trying to get a better understanding of how memcached works. I'm starting with a simple example here to see how this could be handled. The numbers are just taken for example, if I didn't mention this already. When this simple example is ok, more complexity is added and numbers change. Still, I want to start simple and bottom up instead of with a full blown you-can't-handle-this-at-all-example. And also I don't expect mathematically exact numbers but I just want to get an idea from this.
Back to the topic: I created the memcached-session-manager (a tomcat session manager keeping sessions in memcached just for backup / session failover, they're still stored in memory for normal operations; see [1]) and I want to find out how memcached should be "tuned" to provide "best" results. Of course this will be application specific and require some knowledge of memcached. Still I want to be able to give advices. And in the end it would be totally ok for me if the result would be s.th. like "use the max-possible-session-size as min slab size, multiply this with the max-number-of-sessions and this gives you 10% of the memory you need to give memcached". Thanx for your help, cheers, Martin [1] http://code.google.com/p/memcached-session-manager/ On Sat, Mar 13, 2010 at 1:07 AM, Marc Bollinger <[email protected]>wrote: > Part of the disconnect is that, "how do I have to run memcached to > 'store' these sessions in memcached," is not a concrete question. It's > wibbly wobbly at best to try and achieve this behavior, and, "You > can't," _is_ concrete in that there is no way to do this in a > mathematically provable way. The best you're going to get is something > that anecdotally works, contingent on the existence of roughly > homogeneous object sizes, and that you're allocating enough memory to > memcached. The scheme you described above (maybe tweak the growth > factor downward to taste?) will probably work, but that assumes a > limit of 1000 users, which is unrealistic for the scale that memcached > was really designed for in the first place. > > - Marc > > On Fri, Mar 12, 2010 at 3:56 PM, Martin Grotzke > <[email protected]> wrote: > > Hi Brian, > > you're making a very clear point. However it would be nice if you'd > provide > > concrete answers to concrete questions. I want to get a better > understanding > > of memcached's memory model and I'm thankful for any help I'm getting > here > > on this list. If my intro was not supporting this please forgive me... > > Cheers, > > Martin > > > > On Sat, Mar 13, 2010 at 12:27 AM, Brian Moon <[email protected]> wrote: > >> > >> The resounding answer you will get from this list is: You don't, can't > and > >> won't with memcached. That is not its job. It will never be its job. > Perhaps > >> when storage engines are done, maybe. But then you won't get the > performance > >> that you get with memcached. There is a trade off for performance. > >> > >> Brian. > >> -------- > >> http://brian.moonspot.net/ > >> > >> On 3/12/10 3:02 PM, martin.grotzke wrote: > >>> > >>> Hi, > >>> > >>> I know that this topic is rather burdened, as it was said often enough > >>> that memcached never was created to be used like a reliable datastore. > >>> Still, there are users interested in some kind of reliability, users > >>> that want to store items in memcached and be "sure" that these items > >>> can be pulled from memcached as long as they are not expired. > >>> > >>> I read the following on memcached's memory management: > >>> "Memcached has two separate memory management strategies: > >>> - On read, if a key is past its expiry time, return NOT FOUND. > >>> - On write, choose an appropriate slab class for the value; if it's > >>> full, replace the oldest-used (read or written) key with the new one. > >>> Note that the second strategy, LRU eviction, does not check the expiry > >>> time at all." (from "peeping into memcached", [1]) > >>> > >>> I also found "Slabs, Pages, Chunks and Memcached" ([2]) a really good > >>> explanation of memcached's memory model. > >>> > >>> Having this as background, I wonder how it would be possible to get > >>> more predictability regarding the availability of cached items. > >>> > >>> Asume that I want to store sessions in memcached. How could I run > >>> memcached so that I can be sure that my sessions are available in > >>> memcached when I try to "get" them? Additionally asume, that I expect > >>> to have 1000 sessions at a time in max in one memcached node (and that > >>> I can control/limit this in my application). Another asumption is, > >>> that sessions are between 50kb and 200 kb. > >>> > >>> The question now is how do I have to run memcached to "store" these > >>> sessions in memcached? > >>> > >>> Would it be an option to run memcached with a minimum slab size of > >>> 200kb. Then I would know that for each session a 200kb chunk is used. > >>> When I have 1000 session between 50kb and 200kb this should take 200mb > >>> in total. When I run memcached with more than 200mb memory, could I be > >>> sure, that the sessions are alive as long as they are not expired? > >>> > >>> What do you think about this? > >>> > >>> Cheers, > >>> Martin > >>> > >>> > >>> [1] > >>> http://blog.evanweaver.com/articles/2009/04/20/peeping-into-memcached/ > >>> [2] > >>> http://www.mikeperham.com/2009/06/22/slabs-pages-chunks-and-memcached/ > > > > > > > > -- > > Martin Grotzke > > http://www.javakaffee.de/blog/ > > > -- Martin Grotzke http://www.javakaffee.de/blog/
