Comment #8 on issue 202 by [email protected]: TOP_KEYS feature fixes
http://code.google.com/p/memcached/issues/detail?id=202
OK! When you have a moment please try this:
https://github.com/sflow-nhm/memcached
./configure --enable-sflow
I forked from the "engine" branch, and added sFlow support. There is no
additional locking in the critical path, so this should have almost no
impact on performance provided the sampling 1-in-N is chosen sensibly.
(Please test and confirm!)
In addition to cluster-wide top-keys, missed-keys etc. you also get
microsecond-resolution response-time measurements; the value-size in bytes
for each sampled operation and the layer-4 socket. So the sFlow collector
may choose to correlate results by client IP/subnet/country as well as by
cluster node or any function of the sampled keys.
The logic is best described by the daemon/sflow_mc.h file where the steps
are captured as macros so that they can be inserted in the right places in
memcached.c with minimal source-code footprint. The sflow_sample_test() fn
is called at the beginning of each ascii or binary operation, and it
tosses a coin to decide whether to sample that operation. If so, it just
records the start time. At the end of the transaction, if the start_time
was set then the sFlow sample is encoded and submitted to be sent out.
To configure for 1-in-5000, edit /etc/hsflowd.auto to look like this:
rev_start=1
polling=30
sampling.memcache=5000
agentIP=10.211.55.4
collector=127.0.0.1 6343
rev_end=1
Inserting correct IP address for agentIP.
If you compile and run "sflowtool" from the sources, you should see the
ascii output:
http://www.inmon.com/technology/sflowTools.php
For more background and a simple example, see here:
http://blog.sflow.com/2010/10/memcached-missed-keys.html
The periodic sFlow counter-export is not working yet (that's what the
polling=30 setting is for). I think the default-engine needs to implement
the .get_stats_block API call before that will work. Let me know if you
want me to try adding that.
Best Regards,
Neil
P.S. I did try to do this as an engine-shim, but the engine protocol is
really a different, internal, protocol. There was not a 1:1
correspondence with the standard memcached operations.
