[
https://issues.apache.org/jira/browse/CASSANDRA-19596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17924743#comment-17924743
]
Ariel Weisberg edited comment on CASSANDRA-19596 at 2/7/25 4:05 PM:
--------------------------------------------------------------------
This doesn't yield the improvement I was hoping for. I was expecting 10x, but
it turns out the redundant sorts are pretty fast. Possibly because the data is
already sorted, possibly because the smaller sorts are faster. Still
{{71770ms}} for {{trunk}} and {{43258ms}} with 19596. Still worth doing, but
it's not "out of the woods" territory.
CASSANDRA-20158 will help a lot with early open which repeatedly replaces
sstable readers. 20164 might help a little, but I am guessing that compaction
removes a bunch of readers and then adds some at the same time and that is
still going to take the slow path of rebuilding stables. Flushing will get a
little faster since it just adds an sstable.
was (Author: aweisberg):
This doesn't yield the improvement I was hoping for. I was expecting 10x, but
it turns out the redundant sorts are pretty fast. Possibly because the data is
already sorted, possibly because the smaller sorts are faster. Still
{{71770ms}} for {{trunk}} and {{43258ms}} with 19596. Still worth doing, but
it's not "out of the woods" territory.
20158 will help a lot with early open which repeatedly replaces sstable
readers. 20164 might help a little, but I am guessing that compaction removes a
bunch of readers and then adds some at the same time and that is still going to
take the slow path of rebuilding stables. Flushing will get a little faster
since it just adds an sstable.
> IntervalTree build throughput is low enough to be a bottleneck
> --------------------------------------------------------------
>
> Key: CASSANDRA-19596
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19596
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Local/Compaction, Local/SSTable
> Reporter: Ariel Weisberg
> Assignee: Ariel Weisberg
> Priority: Normal
> Fix For: 5.x
>
>
> With several terabytes of data and 8 compactors it’s possible for the
> compactors to spend a lot of time blocked waiting on IntervalTrees to be
> built.
> There is also a lot of wasted CPU because it’s updated optimistically so most
> of them end up being thrown away.
> This can end up being quite painful because it can block memtable flushing as
> well and then a single slow CFS can block unrelated CFS because the memtable
> post flush executor is single threaded and shared across all CFS.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
