Tiny bit of clarification/questions below.
On Tuesday, September 27, 2011 3:25:18 PM UTC-7, Rohit Karlupia wrote:
> - Memory management is not slab based. It is self tuning. Just tell how
much memory to use.
Looks like you've got a few memory allocator options in there. The issue in
most of them is what happens when you run the thing with a heavy production
load for several days. The slab allocator won't fragment. I've heard similar
things about jemalloc in really large production installs, but in a "normal"
application scenario. It's much tougher in memcached because of the
following:
> - LRU is not slab based. It is global. Always the LRU entry is deleted,
irrespective of its size.
If your LRU isn't slab based, how do you free the right amount of memory for
the incoming data?
For example, if you load up tons and tons of 13 byte objects and then
suddenly you need to store a 1MB object, an LRU that *can* free up the space
for you can't do it within any kind of time bounds. Freeing up ~81k LRU
items to get to 1MB is not terribly likely to get you a 1MB contiguous
memory block. The slab will always do so.
Also, from your README:
"For example when using slab sizes of 64/128/256/512/1024 bytes object
with size 513 is stored in 1024 bytes slab, wasting almost half the memory."
You'd have to intentionally mistune it to get it to use those sizes. By
default, 513 bytes is stored in 600 bytes (which leaves 87 bytes for your
key, flags, and expiration). The slabber can be tweaked to use those sizes,
but you'd only do so if it actually benefitted your application.
> - It is scriptable using LUA. What this means is that instead of being
restricted to set, lists
> and other predefined data structures exposed via redis, new data
structures can be created
> and used. Currently I have implemented set, map, quota and sliding window
counter in lua.
> New objects can be implemented without touching the c source code.
This is pretty awesome. Have you considered just building it as an
engine? Then you'd also get the binary protocol, IPv6, UDP, domain sockets,
threading, etc... and any future bug fixes for free.
> The interface for accessing scriptable objects is implemented via
memcached get requests.
> For example: get set:new:mykey - would create a new set object referred
via myKey
Those are valid script keys. Why not "script set:new:mykey" ?
> I ran some tests on my laptop for comparing the HIT rate of cacheismo vs
memcached.
> This post has a graph which shows the difference.
> http://chakpak.blogspot.com/2011/09/introducing-cacheismo.html
Do you have more info on how you built this? I'd be interested to see the
actual rates you were getting on which version of memcached with what
configuration -- specifically, what you were doing and what you saw when
cacheismo was 80% faster.
