Ariel Weisberg created CASSANDRA-12071:
------------------------------------------
Summary: Regression in flushing throughput under load after
CASSANDRA-6696
Key: CASSANDRA-12071
URL: https://issues.apache.org/jira/browse/CASSANDRA-12071
Project: Cassandra
Issue Type: Bug
Components: Local Write-Read Paths
Reporter: Ariel Weisberg
The way flushing used to work is that a ColumnFamilyStore could have multiple
memtables flushing at once. The way it works now there can be only a single
flush of any memtable running in the C* process, and the number of threads
applied to that flush is bounded by the number of disks in JBOD.
This works ok most of the time but occasionally flushing will be a little
slower and ingest will outstrip it and then block on available memory. At this
point you see several second stalls that cause timeouts.
This is a problem for reasonable configurations that don't use JBOD but have
access to a fast disk that can handle some IO queuing (RAID, SSD).
You can reproduce on beefy hardware (12 cores 24 threads, 64 gigs of RAM, SSD)
if you unthrottle compaction or set it to something like 64 megabytes/second
and run with 8 compaction threads and stress with the default write workload
and a reasonable number of threads. I tested with 96.
It started happening after about 60 gigabytes of data was loaded.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
