I didn't know about this cfhistograms thing, very nice!
From: user@cassandra.apache.org
Subject: Re: Unexplained query slowness
Have a look at your column family histograms (nodetool cfhistograms iirc), if
you notice things like a very long tail, a double hump or outliers it would
indicate
Our Cassandra database just rolled to live last night. I’m looking at our query
performance, and overall it is very good, but perhaps 1 in 10,000 queries takes
several hundred milliseconds (up to a full second). I’ve grepped for GC in the
system.log on all nodes, and there aren’t any recent GC
You can use query tracing to check what is happening. Also you fire
jconsole/JavaVisualVM and push out some metrics like the 99th read Beans
for that column family.
A simpler check is using cfstats and look for weird numbers (high number
sstables, if you are deleting check how much tombstones per