[
https://issues.apache.org/jira/browse/CASSANDRA-8301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14227931#comment-14227931
]
Nikolai Grigoriev commented on CASSANDRA-8301:
----------------------------------------------
Oh...I think I see what you mean. I only created the situation where nothing
overlaps at each level - but I've done nothing to respect this rule about the
target number of overlapping sstables between the levels. So, if I understand
correctly, this will (or may - depending on how lucky I am) result in slower
promotion of the sstables to the upper levels, right?
Yes, I was checking the logs carefully to see the result of my manipulations -
the only error I saw was about the keys out-of-order in a single sstable file -
this could not be caused by my re-leveling.
What I observe now is that the remaining ~280 pending compactions go very
slowly, there is quite a bit of sstables at level 0. Under normal traffic this
number seems to be floating around ~600 and probably even increasing. Each
compaction grabs some but while it is working new ones get created :) I think
new sstables get created a bit faster than they are compacted and promoted.
Could it be due to bad leveling?
Regarding initial set of L0 sstables...In my case I had 79 sstables with the
token ranges like "-9010847458915378120,9190536470441980462". I believe they
are original L0 sstables from other machines. I think for those there is no
choice but to put them in L0, otherwise they will overlap with all other
sstables.
I think I will try to implement that algorithm differently. So, just to confirm
I get it right:
- no overlap allowed at any level except L0
- for each sstable at level N there should be no more than 10 at level N+1
- anything that does not fit goes to L0
- sstables with large token ranges have to go to L0 anyway
Interesting...these first two rules most likely create a number of different
possible combinations.
> Create a tool that given a bunch of sstables creates a "decent" sstable
> leveling
> --------------------------------------------------------------------------------
>
> Key: CASSANDRA-8301
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8301
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Marcus Eriksson
>
> In old versions of cassandra (i.e. not trunk/3.0), when bootstrapping a new
> node, you will end up with a ton of files in L0 and it might be extremely
> painful to get LCS to compact into a new leveling
> We could probably exploit the fact that we have many non-overlapping sstables
> in L0, and offline-bump those sstables into higher levels. It does not need
> to be perfect, just get the majority of the data into L1+ without creating
> overlaps.
> So, suggestion is to create an offline tool that looks at the range each
> sstable covers and tries to bump it as high as possible in the leveling.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
