Thanks Dormando, that was perfect. My apologies Matt, you gave me the correct information had I been running 1.4.x, but I didn't see those stats in 1.2.8. Makes much more sense now.
Keep in mind guys, I'm not trying to disparage memcache, just trying to make sure I understand and can track all of the variables at play here. Last thing Dormando, would you agree that it's not possible to compute this information without the mem_requested stat? On Dec 4, 3:32 pm, dormando <[email protected]> wrote: > > Thanks Matt. I did look a bit through the 'stats slabs' command, but > > perhaps I'm not interpreting it correctly. In the most basic test, > > when I put in a 100 byte object, I'm seeing a slab being created with > > a chunk size of 176. However, 'stats sizes' shows me one item of > > '192'. So there's part of my confusion...am I using 176 bytes for the > > object or 192? > > > The second part of my confusion is the ability to actually see that > > 100 byte object. If instead of 100 bytes, I use 150, I'm not seeing > > any difference in the output of 'stats slabs' or 'stats sizes'. > > Obviously I can do these contrived tests and know what it is I'm > > putting into the cache, but I'm concerned that when it moves into a > > production setting I won't know the exact size of all the objects in > > the cache. I'm using server version 1.2.8 at the moment. > > > Am I reading these stats incorrectly? > > > Any detailed help would be really appreciated. > > > Thanks so much. > > An item size is the value length + the key length + pointers + bytes to > store the length + CAS header + a couple terminators/misc things. I don't > have the exact item overhead offhand but will look it up and put it in the > wiki. > > You can easily calculate your memory overhead on a slab: > > STAT 3:chunk_size 152 > STAT 3:chunks_per_page 6898 > STAT 3:total_pages 291 > STAT 3:total_chunks 2007318 > STAT 3:used_chunks 2007310 > STAT 3:free_chunks 8 > STAT 3:free_chunks_end 0 > STAT 3:mem_requested 271013713 > > chunk_size * chunks_per_page is the amount of bytes in a page for this > slab class, which is 1048496 here. > > * 291 pages == 305112336 bytes allocated in the slab. > > mem_requested is a shorthand that states the amount of memory actual items > (the total length, value + key + misc) take up within a slab. > > 271013713 / 305112336 > ~0.89% rounded. > > So I've lost 11% memory overhead in this slab on top of the ~30 bytes per > item. used_chunks * the standard overhead will give you most of the rest > of the memory overhead. So it's probably closer to 60 megabytes, total? > > 'stats sizes' will throw items into the nearest "slab" if everything were > cut by 64 byte slabs. The rounding is probably putting it into the 192 > byte bucket for you. If your item goes into the 172 byte slab, you're > definitely using 172 bytes or less. > > The idea is that we trade off some memory overhead for consistent speed in > O(1) operations. We know ways to improve the efficiency and will be doing > so over the next few months, but I wouldn't say this is horrific at all. > Remove some headers, switch back to malloc or jemalloc/etc and you lose > consistent performance. > > The overhead is most pronounced for small keys as well. Consider reducing > your key size, disabling CAS (-C) if you never use it (8 bytes per item), > or reducing the slab growth factor to close down the overhead. > > As soon as I get a chance I'm adding some more modes to damemtop so folks > can more easily see slab overhead... The mem_requested stat trond added > and the pointer_size stat lets us trivially calculate overhead given that > you already understand that a stored "value" is actually "key + value" for > length, not just the value you're storing. > > I'll throw out a side pointer here actually; This sort of knowledge is why > it's nice that memcached can store 0 byte values. If your client allows > you, you can store bits in the flags section, otherwise the existence of > the key itself may be enough data for some things you store. > > -Dormando
