I didn't know about this cfhistograms thing, very nice!
From: user@cassandra.apache.org
Subject: Re: Unexplained query slowness
Have a look at your column family histograms (nodetool cfhistograms iirc), if
you notice things like a very long tail, a double hump or outliers it would
indicate something wrong with your data model or you have a hot partition key/s.
Also looking at your 99 and 95 percentile latencies will just hide these
occasional high latency reads as they fall outside these percentiles.
If you are running a stock config, first rule out that its not your data model,
then investigate things like disk latency, noisy neighbours (if you are on vms/
in the cloud).
On 26 February 2015 at 03:01, Marcelo Valle (BLOOMBERG/ LONDON)
mvallemil...@bloomberg.net wrote:
I am sorry if it's too basic and you already looked at that, but the first
thing I would ask would be the data model.
What data model are you using (how is your data partitioned)? What queries are
you running? If you are using ALLOW FILTERING, for instance, it will be very
easy to say why it's slow.
Most times people get slow queries in Cassandra they are using the wrong data
model.
[]s
From: user@cassandra.apache.org
Subject: Re:Unexplained query slowness
Our Cassandra database just rolled to live last night. I’m looking at our query
performance, and overall it is very good, but perhaps 1 in 10,000 queries takes
several hundred milliseconds (up to a full second). I’ve grepped for GC in the
system.log on all nodes, and there aren’t any recent GC events. I’m executing
~500 queries per second, which produces negligible load and CPU utilization. I
have very minimal writes (one every few minutes). The slow queries are across
the board. There isn’t one particular query that is slow.
I’m running 2.0.12 with SSD’s. I’ve got a 10 node cluster with RF=3.
I have no idea where to even begin to look. Any thoughts on where to start
would be greatly appreciated.
Robert
--
Ben Bromhead
Instaclustr | www.instaclustr.com | @instaclustr | (650) 284 9692
