Our Cassandra database just rolled to live last night. I’m looking at our query performance, and overall it is very good, but perhaps 1 in 10,000 queries takes several hundred milliseconds (up to a full second). I’ve grepped for GC in the system.log on all nodes, and there aren’t any recent GC events. I’m executing ~500 queries per second, which produces negligible load and CPU utilization. I have very minimal writes (one every few minutes). The slow queries are across the board. There isn’t one particular query that is slow.
I’m running 2.0.12 with SSD’s. I’ve got a 10 node cluster with RF=3. I have no idea where to even begin to look. Any thoughts on where to start would be greatly appreciated. Robert