Hi Dormando, thanks for suggestions, background thread would be nice...
The idea is actually that with 2-3GB i get plenty of evictions of items
that need to be fetched later. And with 16GB i still get evictions,
actually probably i could throw more memory than 16G and it'd only result
in more expired items sitting in the middle of slabs, forever... Now im
going for persistence. Sounds probably crazy, but we're having some data
that we can't loose:
1. statistics, we aggregate writes to DB using memcached (+list
implementation). If these items get evicted we're loosing rows in db.
Loosing data sometimes isn't a big problem. Eg. we restart memcached once a
week so we're loosing 1 minute of data every week. But if we have evictions
we're loosing data constantly (which we can't have)
2. we drive load balancer using data in memcached for statistics, again,
not nice to loose data often because workers can get incorrect amount of
traffic.
3. we're doing some adserving optimizations, eg. counting per-domain ad
priority, for one domain it takes about 10 seconds to analyze all data and
create list of ads, so can't be done online... we put result of this in
memcached, if we loose too much of this the system will start to serve
suboptimal ads (because it'll need to switch to more general data or much
simpler algorithm that can be done instantly)
Probably would be best to rewrite all this using C or golang, and use
memcached just for caching, but it'd take too much time which we don't have
currently...
I have seen twitter and nk implementations that seem to do what i need, but
they seem old (based on old code), so I prefer to modify code of recent
"official" memcached, to not be stuck with old code or abandonware.
Actually there are many topics about limitations of currrent eviction algo
and option to enable some background thread to do scraping based on
statistics of most filled slabs (with some parameter to specify if it
should take light or aggressive approach) would be nice...
As for the code... is that slab_rebalance_move function in slab.c? It seems
a little difficult to gasp without some DOCs of how things are working...
can you please write a very short description of how this "angry birds"
more workd?
I have quick question about this above... linked is item that's placed on
linked list, but what other flags means, and why 2 last are 2 of them
temporary?
#define ITEM_LINKED 1
#define ITEM_CAS 2
/* temp */
#define ITEM_SLABBED 4
#define ITEM_FETCHED 8
This from slab_rebalance_move seems interesting:
refcount = refcount_incr(&it->refcount);
...
if (refcount == 1) { /* item is unlinked, unused */
...
} else if (refcount == 2) { /* item is linked but not busy */
Is there some docs about refcounts, locks and item states? Basically why
item with refcount 2 is not busy? You're increasing refcount by 1 on
select, then again when reading data? Can refcount ever be higher than 2 (3
in above case), meaning 2 threads can access same item?
Thanks.
W dniu czwartek, 10 kwietnia 2014 06:05:30 UTC+2 użytkownik Dormando
napisał:
>
> > Hi Guys,
> > im running a specific case where i don't want (actually can't have) to
> have evicted items (evictions = 0 ideally)... now i have created some
> simple
> > algo that lock the cache, goes through linked list and evicts items...
> it makes some problems, like 10-20ms cache locks on some cases.
> >
> > Now im thinking about going through each slab memory (slabs keep a list
> of allocated memory regions) ... looking for items, if expired item is
> > found, evict it... this way i can go eg. 10k items or 1MB of memory at a
> time + pick slabs with high utilization and run this "additional" eviction
> > only on them... so it'll prevent allocating memory just because unneded
> data with short TTL is occupying HEAD of the list.
> >
> > With this linked list eviction im able to run on 2-3GB of memory...
> without it 16GB of memory is exhausted in 1-2h and then memcached starts to
> > kill "good" items (leaving expired ones wasting memory)...
> >
> > Any comments?
> > Thanks.
>
> you're going a bit against the base algorithm. if stuff is falling out of
> 16GB of memory without ever being utilized again, why is that critical?
> Sounds like you're optimizing the numbers instead of actually tuning
> anything useful.
>
> That said, you can probably just extend the slab rebalance code. There's a
> hook in there (which I called "Angry birds mode") that drives a slab
> rebalance when it'd otherwise run an eviction. That code already safely
> walks the slab page for unlocked memory and frees it; you could edit it
> slightly to check for expiration and then freelist it into the slab class
> instead.
>
> Since it's already a background thread you could further modify it to just
> wake up and walk pages for stuff to evict.
>
--
---
You received this message because you are subscribed to the Google Groups
"memcached" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
