On Mon, 14 Nov 2016, Bill Moseley wrote: > Thanks for the response. > > Sorry for all the questions. I'm trying to fully understand before upgrading. > On Sun, Nov 13, 2016 at 11:52 PM, dormando <dorma...@rydia.net> wrote: > > > * If all items in the slab are reclaimed then "stats items" no > longer shows that slab. I wonder if that is correct because we can no longer > see stats on evicted > items for > > that slab. > > It's always been that way... just never fixed/changed it I guess? > > > I guess in older versions I never saw it disappear since we always did a > "set" which would then evict/replace an item and the slab would never be > empty. With rebalancing it's > possible for all items to be removed from the slab. > (But the counters like "evicted" are still maintained.) > > > It doesn't appear that rebalancing works within a slab, correct? > > That is, if I run with "-m 5 -o modern" and fill a 100k-item slab it will say > "total_pages 5". If all the items expire in 60 seconds then after > rebalancing it will then show > "total_pages 2" and "slab_global_page_pool 3". > > If I do the same thing again, but this time I alternate the items' expires > with 60 and 0 and wait 60 seconds for the rebalance to remove items then it > sill shows "total_pages > 5" and "slab_global_page_pool 0". > > The rebalancing won't shuffle items from one page to another page in a given > slab to fully free up a page, correct? > > In other words, If there's a single item with an expires of zero (never > expire) stored in a page then that page won't ever be freed back to the > slab_global_page_pool (unless > that item is force evicted). > > Am I understanding that correctly?
No, it does make a best-effort attempt to shuffle memory so it can be freed. You probably need to test with more than 5 slabs though. It will take free chunks that exist outside of the slab page it currently wants to free, and moves valid items into different areas of memory. It'll shuffle for a bit and then free up that slab page if it can. Just might require a bit more memory in active use (and then freed up via expirey) before it's able to do that dance safely. > > And yes, we do sometimes store items > 1mb. Currently we are not using -I > and do the chunking into parts (and maybe gzipping) in the client before > storing. So, we store one > or more 1mb items and then the remainder in another chunk based on its size. > > If we use new large item support all of the split-up parts of the item we are > storing end up in the same chunk. Does that mean a single overflow > (remainder) byte would use up > another full chunk-size? (524288 bytes?) Yup. It's on average more efficient than the old method, by quite a lot. Once I get some time to refactor the first pass feature it should be able to "cap" an item with a chunk from a smaller slab class. (so 512k + 100 bytes or something) my goal then would be to lower the chunk max to somewhere in 16-64k range, which should compact the smaller slab classes closer together, and generally raise memory efficiency. No timeline for this, sadly. > > Thanks, > > -- > Bill Moseley > mose...@hank.org > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
