[
https://issues.apache.org/jira/browse/CASSANDRA-4310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396529#comment-13396529
]
sankalp kohli commented on CASSANDRA-4310:
------------------------------------------
What you are saying is true. But the improvement I am saying has more than
this. It also does compactions in parallel between different levels and also
multiple compactions per level. So it will definitely speed things up. It is
quite frustrating to see Disk not being fully used when you are using SSD.
Also like you said L0->L1 is the biggest bottleneck. This will help it in a
way. So when L0(32 stable) gets merged with L1, then L1 will merge with L2 and
so on. But with this, you will be doing L0-L1 compactions almost every cycle
unless L1-L2 is happening. So when say L3 -> L4, L0->L1 compaction won't happen
when it can.
So this solution cannot help parallelize L0->L1, but it will help since it runs
L0->L1 almost every time. It does not get blocked by compactions in higher
levels.
> Multiple independent Level Compactions in Parallel(Useful for SSD).
> --------------------------------------------------------------------
>
> Key: CASSANDRA-4310
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4310
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 1.1.1, 1.1.2
> Reporter: sankalp kohli
> Labels: compaction, features, leveled, performance, ssd
>
> Problem: If you are inserting data into cassandra and level compaction cannot
> catchup, you will create lot of files in L0.
> Here is a solution which will help here and also increase the performance of
> level compaction.
> We can do many compactions in parallel for unrelated data.
> 1) For no over lapping levels. Ex: when L0 stable is compacting with L1, we
> can do compactions in other levels like L2 and L3 if they are eligible.
> 2) We can also do compactions with files in L1 which are not participating in
> L0 compactions.
> This is specially useful if you are using SSD and is not bottlenecked by IO.
> I am seeing this issue in my cluster. The compactions pending are more than
> 50k and the disk usage is not that much(I am using SSD).
> I am doing multithreaded to true and also not throttling the IO by putting
> the value as 0.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
