Patrick McFadin created CASSANDRA-21749:
-------------------------------------------
Summary: Flush during compaction can bring back deleted data
Key: CASSANDRA-21749
URL: https://issues.apache.org/jira/browse/CASSANDRA-21749
Project: Apache Cassandra
Issue Type: Bug
Components: Local/Compaction
Reporter: Patrick McFadin
Compaction decides whether a tombstone can be dropped by checking which other
sstables might hold older data for the same partition. That list is built once,
when the compaction starts. If older data for the partition is flushed from a
memtable, streamed in, or imported while the compaction is running, compaction
doesn't see it. It drops the tombstone, and the deleted data comes back.
It needs a tombstone past gc_grace_seconds, and older-timestamped data arriving
during the compaction. Repair, hints and client-supplied timestamps can all
deliver such data.
h3. Reproduce
# Table with gc_grace_seconds = 0. Delete partition k1 at ts=10 and flush
(sstable A).
# Write k1 at ts=1. It goes to the memtable and is hidden by the tombstone.
# Start a compaction of A.
# While it runs, flush the memtable.
# k1 is readable again after the compaction.
If the flush happens before the compaction starts, k1 stays deleted.
A deterministic unit test (CompactionControllerTest style) and a cqlsh repro
are attached.
h3. Cause
{{CompactionController}} computes its overlapping sstables in the constructor
and only refreshes them when one of those sstables gets compacted away
({{maybeRefreshOverlaps}}). Sstables that are newly added to the live set are
never picked up. {{getPurgeEvaluator}} checks that stale set plus the current
memtables, so a memtable flushed after the controller was created is in neither.
h3. Fix
Record the live sstable set when the overlaps are computed, and refresh the
overlaps in {{getPurgeEvaluator}} (and {{getFullyExpiredSSTables}}) whenever
the live set has changed. In the common case this costs one reference
comparison per partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
