[
https://issues.apache.org/jira/browse/CASSANDRA-14281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16426970#comment-16426970
]
Michael Burman commented on CASSANDRA-14281:
--------------------------------------------
{code:java}
trunk:
op rate : 44,168 op/s [WRITE: 44,168 op/s]
partition rate : 44,168 pk/s [WRITE: 44,168 pk/s]
row rate : 44,168 row/s [WRITE: 44,168 row/s]
latency mean : 2.9 ms [WRITE: 2.9 ms]
latency median : 2.0 ms [WRITE: 2.0 ms]
latency 95th percentile : 7.0 ms [WRITE: 7.0 ms]
latency 99th percentile : 11.8 ms [WRITE: 11.8 ms]
latency 99.9th percentile : 115.8 ms [WRITE: 115.8 ms]
latency max : 191.9 ms [WRITE: 191.9 ms]
latency_writes branch:
op rate : 46,633 op/s [WRITE: 46,633 op/s]
partition rate : 46,633 pk/s [WRITE: 46,633 pk/s]
row rate : 46,633 row/s [WRITE: 46,633 row/s]
latency mean : 2.7 ms [WRITE: 2.7 ms]
latency median : 1.9 ms [WRITE: 1.9 ms]
latency 95th percentile : 6.3 ms [WRITE: 6.3 ms]
latency 99th percentile : 11.1 ms [WRITE: 11.1 ms]
latency 99.9th percentile : 112.1 ms [WRITE: 112.1 ms]
latency max : 212.5 ms [WRITE: 212.5 ms]
{code}
> Improve LatencyMetrics performance by reducing write path processing
> --------------------------------------------------------------------
>
> Key: CASSANDRA-14281
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14281
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Michael Burman
> Assignee: Michael Burman
> Priority: Major
>
> Currently for each write/read/rangequery/CAS touching the CFS we write a
> latency metric which takes a lot of processing time (up to 66% of the total
> processing time if the update was empty).
> The way latencies are recorded is to use both a dropwizard "Timer" as well as
> "Counter". Latter is used for totalLatency and the previous is decaying
> metric for rates and certain percentile metrics. We then replicate all of
> these CFS writes to the KeyspaceMetrics and globalWriteLatencies.
> Instead of doing this on the write phase we should merge the metrics when
> they're read. This is much less common occurrence and thus we save a lot of
> CPU time in total. This also speeds up the write path.
> Currently, the DecayingEstimatedHistogramReservoir acquires a lock for each
> update operation, which causes a contention if there are more than one thread
> updating the histogram. This impacts scalability when using larger machines.
> We should make it lock-free as much as possible and also avoid a single
> CAS-update from blocking all the concurrent threads from making an update.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
