[
https://issues.apache.org/jira/browse/CASSANDRA-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721586#comment-14721586
]
Diogo Guerra commented on CASSANDRA-5371:
-----------------------------------------
I'm benchmarking our solution that uses LCS with C* 2.1.8 and we have the
scenario here by [~jjordan] every week we need to burst C* with a batch job
that takes 18-20h. The system behaves very well dureing the whole week, but
during that time degrades considerably on the highest percentiles (>p99) were
we get reads rocketing to latencies > 200ms.
I'm working on understanding what is happening with the system since I don't
see IO or CPU exhausted. Wonder what is the usual rate of compaction with LCS.
In my system log I see that compaction tasks are only doing 2-3MB/s
{code}
INFO [CompactionExecutor:93] 2015-08-30 17:06:22,114 CompactionTask.java:274
- Compacted 9 sstables to
[/mnt/ssd/cassandra/data/pulse/metrics-ea89aeb0465e11e586e7575910f56afe/pulse-metrics-ka-20266,/mnt/ssd/cassandra/data/pulse/metrics-ea89aeb0465e11e586e7575910f56afe/pulse-metrics-ka-20274,/mnt/ssd/cassandra/data/pulse/metrics-ea89aeb0465e11e586e7575910f56afe/pulse-metrics-ka-20280,/mnt/ssd/cassandra/data/pulse/metrics-ea89aeb0465e11e586e7575910f56afe/pulse-metrics-ka-20286,/mnt/ssd/cassandra/data/pulse/metrics-ea89aeb0465e11e586e7575910f56afe/pulse-metrics-ka-20291,/mnt/ssd/cassandra/data/pulse/metrics-ea89aeb0465e11e586e7575910f56afe/pulse-metrics-ka-20297,/mnt/ssd/cassandra/data/pulse/metrics-ea89aeb0465e11e586e7575910f56afe/pulse-metrics-ka-20302,/mnt/ssd/cassandra/data/pulse/metrics-ea89aeb0465e11e586e7575910f56afe/pulse-metrics-ka-20307,].
1,383,544,305 bytes to 1,335,157,959 (~96% of original) in 450,225ms =
2.828154MB/s. 153,450 total partitions merged to 130,214. Partition merge
counts were {1:106978, 2:23236, }
{code}
Is this normal?
> Perform size-tiered compactions in L0 ("hybrid compaction")
> -----------------------------------------------------------
>
> Key: CASSANDRA-5371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5371
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 2.0 beta 1
>
> Attachments: HybridCompactionStrategy.java
>
>
> If LCS gets behind, read performance deteriorates as we have to check bloom
> filters on man sstables in L0. For wide rows, this can mean having to seek
> for each one since the BF doesn't help us reject much.
> Performing size-tiered compaction in L0 will mitigate this until we can catch
> up on merging it into higher levels.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
