[
https://issues.apache.org/jira/browse/CASSANDRA-7019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095937#comment-15095937
]
Branimir Lambov commented on CASSANDRA-7019:
--------------------------------------------
My understanding of the problem is that since we only remove deleted content
when we compact together the data and its tombstone, there is not much chance
tombstones or data can actually be removed without a global (major) compaction.
This is far from ideal for levelled, and also a problem for any other
compaction method due to the size of the operation.
To enable smaller-sized transformations that do delete data (which would later
enable the removal of tombstones as well), I made a [work-in-progress
patch|https://github.com/blambov/cassandra/tree/7019-tombstone-compaction] of
an option to use all overlapping sstables as sources of tombstones when doing
compaction. These tombstones are used only to filter out deleted content, and
will not appear in the resulting tables. The process will slow the compactions
to which it is applied down as it does have to read from a lot of sstables, but
hopefully the slowdown will be acceptable with the 8099 improvements.
The patch is uploaded with that option turned permanently on (for testing). I
am not yet sure how best to use it, but I can see several possibilities:
- as an option that can be generally turned on (via -D and yaml option, best
via JMX as well) for a node and applies to all compactions. This will trade
compaction performance for improved dataset size and read performance and may
not be suitable for all scenarios.
- as an option that is turned on by strategy, e.g. specifically for the
levelled compaction strategy.
- as a flag that can be turned on when applying specific compactions (e.g.
cleanup).
- as a new compaction type that is applied to one sstable at a time to remove
deleted content from that table.
- as above, but triggered automatically when a tombstone compaction identifies
tables with deleted content.
I personally think the first option with a default value of "on" for the
levelled strategy makes most sense, but I want to hear other opinions.
[~krummas], could you take a look and tell me what you think of the approach
and code, and which of the above makes most sense to you?
> Improve tombstone compactions
> -----------------------------
>
> Key: CASSANDRA-7019
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7019
> Project: Cassandra
> Issue Type: Improvement
> Components: Compaction
> Reporter: Marcus Eriksson
> Assignee: Branimir Lambov
> Labels: compaction
> Fix For: 3.x
>
>
> When there are no other compactions to do, we trigger a single-sstable
> compaction if there is more than X% droppable tombstones in the sstable.
> In this ticket we should try to include overlapping sstables in those
> compactions to be able to actually drop the tombstones. Might only be doable
> with LCS (with STCS we would probably end up including all sstables)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
