You really do want automatic slab reassignment... What you're doing is janky and doesn't really solve the problem. If anything happens that ends up filling more of your cache, you have to restart memcached anyway.
I'm not entirely sure why you're going through all of this trouble, instead of just restarting them occasionally when your hit rate starts to suffer due to changes in cache size? You're overcomplicating the problem. 1.4.0's stats will let you calculate hitrate per slab (on top of being able to monitor evictions per slab, etc). Instead of mass fetching, mass deleting, and doing other generally unscalable things... Do you have proof that your average cache item size changes enough to make this worth it? We *do* need slab reassignment. This doesn't exist presently and isn't a simple or free change, but almost everyone except some very massive sites can get away without needing it. -Dormando On Tue, 21 Jul 2009, dl4ner wrote: > > Hello Trond, > > > The biggest problem with expiration is that in order to locate the > > item to expire we would have to look at _all_ items in the cache > > Yes, I know. That's the reason why > - I suggested an additional command to have it externally triggered > only when needed > (most users should not need it anyway) > - I suggested a deletion limit, for beeing able to have it stopped > after deletion of > the given number of expired keys. (let's say 2.000). the Script can > run more often > anyway. > > that way it could be reasonable fair not to block a thread too long. > btw: that is exactly what I'm doing right now: get a list of 20.000 > items > via stats cachedump, and foreach item do delete if expired. > > > (we don't link the items into a expiry list). > > I understand, and this would not be a problem for me. I also see > that my proposal is only a 90%-solution, but with only 10% of work > needed compared to a perfect solution. > But as 99% of all users would not need that feature anyway, > there should be no problem. But I don't want to patch the source > code with every new version of memcached coming out :) > > > Currently we work out of the assumption that the future memory > > allocation pattern will match the current pattern, so that your items > > sizes wouldn't change that much (and hence the slab distribution > > wouldn't be that off..) > > exactly that is my problem. and we discovered it only after using > memcached for about 18 months... > ok, workaround would be to regularly restart the memcached, so > it can adapt to the a new pattern, but this would also mean a loss > in sessions what I'd like to avoid :) > > > That being said, freeing the item from the slab when it expires would > > not solve your problems completely, it would just postpone the > > problem... > > Yes, completely agreed. But If I have 4 G of RAM, and I normally > would only need about 300 MB, I guess there would easily be enough > additional pages that CAN be allocated if needed. > Then, we can easily check in our network management, whether the > total allocated memory of the memcached process is near the limit > (for example over 3.5 GB) and then step in before it is too late. > I guess it would be some months instead of some days until the > allocated size (pages assigned to slabs) is that big. > I don't have a problem assigning 4 G of RAM even when 500 MB would > do, > if I can avoid that problem this way. 4 G RAM is a lot cheaper than > having > to invent completely new ways inside our application :) > > > You don't know when items expire so a slab page could > > contain items with multiple expiration times, so moving the page to > > another slabclass wouldn't be that easy to do (well, you could move > > items between slab pages etc, but this is not a small patch ;-) ) > > Yes. having that all in mind, I guess a simple hardcoded non-automatic > expire (triggered externally) that goes through the linked item list > and > deletetes every object > + would be a relatively simple patch > + would not affect the normal default behaviour at all > + would not be noticed by any other user at all > + could help to solve our problem for some users (=> at lest for me) > > even if it had some drawbacks (ugly, relatively slow, maybe a small > blocking during the expire run) - but those drawbacks would only be > visible > to those who needed that feature. I for my share would not mind it at > all. > > my client-side expire is running on one cluster (since last friday); > it is started every 30 minutes by now, and helps a lot. the cache size > dropped from about 3 GB to 130 MB, so there is still enough room > Runtime is about 2 minutes expiring two servers, doing an usleep(5) > every 10 > deletes and between every slab. > another cluster did have two servers each 1 G RAM, is now using about > 30-70 M of RAM (memcached-statistics) with about 350 pages allocated. > So it seems to me that it does work (as long as you have enough memory > left to be allocated when needed). > > The performance-impact of my externally triggered expire (php script) > is not reallly noticeable at the memcache-server (load); I guess an > internal > c-function would do a lot faster but could impact the server more, so > it > should have the deletion limit (to be able to limit that impact). > on the other hand, going through a list of some 100.000 items should > be quite fast as everything is in ram. > Even if it would block the memcached for lets say 200 ms... > who can see the difference if the delivered webpage is twice an > hour for 200ms late...? > > regards, > > Werner. >
