[
https://issues.apache.org/jira/browse/CASSANDRA-21749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick McFadin updated CASSANDRA-21749:
----------------------------------------
Severity: Critical
> Flush during compaction can bring back deleted data
> ---------------------------------------------------
>
> Key: CASSANDRA-21749
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21749
> Project: Apache Cassandra
> Issue Type: Bug
> Components: Local/Compaction
> Reporter: Patrick McFadin
> Priority: Urgent
> Fix For: 5.0.x, 6.0.x, 7.x
>
>
> Compaction decides whether a tombstone can be dropped by checking which other
> sstables might hold older data for the same partition. That list is built
> once, when the compaction starts. If older data for the partition is flushed
> from a memtable, streamed in, or imported while the compaction is running,
> compaction doesn't see it. It drops the tombstone, and the deleted data comes
> back.
> It needs a tombstone past gc_grace_seconds, and older-timestamped data
> arriving during the compaction. Repair, hints and client-supplied timestamps
> can all deliver such data.
> h3. Reproduce
> # Table with gc_grace_seconds = 0. Delete partition k1 at ts=10 and flush
> (sstable A).
> # Write k1 at ts=1. It goes to the memtable and is hidden by the tombstone.
> # Start a compaction of A.
> # While it runs, flush the memtable.
> # k1 is readable again after the compaction.
> If the flush happens before the compaction starts, k1 stays deleted.
> A deterministic unit test (CompactionControllerTest style) and a cqlsh repro
> are attached.
> h3. Cause
> {{CompactionController}} computes its overlapping sstables in the constructor
> and only refreshes them when one of those sstables gets compacted away
> ({{maybeRefreshOverlaps}}). Sstables that are newly added to the live set are
> never picked up. {{getPurgeEvaluator}} checks that stale set plus the current
> memtables, so a memtable flushed after the controller was created is in
> neither.
> h3. Fix
> Record the live sstable set when the overlaps are computed, and refresh the
> overlaps in {{getPurgeEvaluator}} (and {{getFullyExpiredSSTables}}) whenever
> the live set has changed. In the common case this costs one reference
> comparison per partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
