[
https://issues.apache.org/jira/browse/CASSANDRA-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316241#comment-14316241
]
Marcus Eriksson commented on CASSANDRA-7409:
--------------------------------------------
bq. I'm a little suspicious of the MOLO=0 results
Me too, looking at the code and running a few short tests indicate that they
are identical in what compaction candidates they pick - we should probably run
a few more tests before committing this.
* LCS.getScanners - different checks if level <= 0 and level <= mol - fold them
up? (The comment above the level <= 0 check is no longer valid - an sstable can
never be in -1)
* When getting candidates for same level compaction, we always start from the
smallest token, should probably record last token we picked and start from that?
* In getCandidatesForSameLevelCompaction, why are we including next level when
level == maxOverlapping? Feels like we should be able to do an in-level
compaction even though the next level is non-overlapping.
* In getCompactionCandidates, what is the reason the score is based on number
of sstables instead of level size in total bytes?
* In getCompactionCandidates, this looks wrong to me: {{Set<SSTableReader>
candidates = Sets.union(Collections.singleton(sstable),
Sets.union(overlapping(sstable, getLevel(level)), overlapping(sstable,
getLevel(level + 1))));}} - We should grab all the sstables in {{level + 1}}
that overlap with any of the sstables we pick from {{level}}. Running stress
against this with an {{assert false;}} in LeveledManifest.add(...) if we cannot
add the sstable will show any overlaps (the only valid case of when this
happens is after an incremental repair and we move sstables from unrepaired
manifest to repaired manifest).
Nits:
* Brace on newline in StorageService.getManifestDescription and
LM.calculateOverlappingScore
* nodetool getmanifest description could be something more generic like "get
compaction manifest"
Random thoughts/future improvements(?):
* What if we made the maximum overlapping level contain as much data as the
first non-overlapping level? Then the sstables would cover approximately the
same ranges and we could probably run more compactions in parallel between
those levels (it would probably increase write amplification though so I'm
unsure if there would be any benefits). We could also, in theory, bump sstables
to higher level without compaction, ie, say that we have maxOverlappingLevel =
2, we run a compaction with one L3-sstable and 10 L4 sstables, this creates a
gap in level 3, and it could be possible to take an L2-sstable and just bump it
up to L3
* Improve the overlap score using CompactionMetadata.cardinalityEstimator (in a
new ticket though)
> Allow multiple overlapping sstables in L1
> -----------------------------------------
>
> Key: CASSANDRA-7409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7409
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Carl Yeksigian
> Assignee: Carl Yeksigian
> Labels: compaction
> Fix For: 3.0
>
>
> Currently, when a normal L0 compaction takes place (not STCS), we take up to
> MAX_COMPACTING_L0 L0 sstables and all of the overlapping L1 sstables and
> compact them together. If we didn't have to deal with the overlapping L1
> tables, we could compact a higher number of L0 sstables together into a set
> of non-overlapping L1 sstables.
> This could be done by delaying the invariant that L1 has no overlapping
> sstables. Going from L1 to L2, we would be compacting fewer sstables together
> which overlap.
> When reading, we will not have the same one sstable per level (except L0)
> guarantee, but this can be bounded (once we have too many sets of sstables,
> either compact them back into the same level, or compact them up to the next
> level).
> This could be generalized to allow any level to be the maximum for this
> overlapping strategy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
