Rob, The 10 seconds latency that I gave earlier is from CQL tracing. Almost 5 seconds out of that was taken up by the “merge memtable and sstables” step. The remaining 5 seconds are from “read live and tombstoned cells.”
I too first thought that maybe disk is not the bottleneck and Cassandra is serving everything from cache, but in that case, it should not take 10 seconds for reading just 20MB data. Also, I narrowed down the query to limit it to a single partition read and I ran the query in cqlsh running on the same node. I turned on tracing, which shows that all the steps got executed on the same node. htop shows that CPU and memory are not the bottlenecks. Network should not come into play since the cqlsh is running on the same node. Is there any performance tuning parameter in the cassandra.yaml file for large reads? Mohammed From: Robert Coli [mailto:rc...@eventbrite.com] Sent: Tuesday, September 16, 2014 5:42 PM To: user@cassandra.apache.org Subject: Re: no change observed in read latency after switching from EBS to SSD storage On Tue, Sep 16, 2014 at 5:35 PM, Mohammed Guller <moham...@glassbeam.com<mailto:moham...@glassbeam.com>> wrote: Does anyone have insight as to why we don't see any performance impact on the reads going from EBS to SSD? What does it say when you enable tracing on this CQL query? 10 seconds is a really long time to access anything in Cassandra. There is, generally speaking, a reason why the default timeouts are lower than this. My conjecture is that the data in question was previously being served from the page cache and is now being served from SSD. You have, in switching from EBS-plus-page-cache to SSD successfully proved that SSD and RAM are both very fast. There is also a strong suggestion that whatever access pattern you are using is not bounded by disk performance. =Rob
