[
https://issues.apache.org/jira/browse/CASSANDRA-18507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17722872#comment-17722872
]
Tobias Lindaaker commented on CASSANDRA-18507:
----------------------------------------------
I have pushed an update that structures the test to be more in line with how
other tests works. This also makes it possible to add other test cases to that
test class in the future (I'd made a mistake in the setup that prevented that
before). I'm not sure if this has any effect on the issue [~dcapwell] had with
running the test or not.
I also incorporated the JVM-DTest that [~marcuse] wrote, since I think it seems
useful to have that level of testing for this as well.
*
[thobe/CASSANDRA-18507-4.0|https://github.com/thobe/cassandra/tree/CASSANDRA-18507-4.0]
*
[thobe/CASSANDRA-18507-4.1|https://github.com/thobe/cassandra/tree/CASSANDRA-18507-4.1]
I have also done some further analysis of the behavior of this test on
{{trunk}}. I said before that the problem does not manifest on {{trunk}}.
However, after further analysis, it seems that is because {{trunk}} uses a
different method for checking for available disk space, so the limiting that's
done in my test (and the limiting don in the JVM-DTest that [~marcuse] wrote)
does not work to trigger a partial compaction, but later there is a different
check that _is_ triggered by the limiting in my test that rejects the
compaction due to lack of disk space. I had misinterpreted this as partial
compactions not being supported at all, but I'll see if I can revise the test
to be applicable to {{trunk}} as well in order to see if this problem also
occurs there.
> Partial compaction can resurrect deleted data
> ---------------------------------------------
>
> Key: CASSANDRA-18507
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18507
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
> Reporter: Tobias Lindaaker
> Assignee: Tobias Lindaaker
> Priority: Normal
>
> If there isn't enough disk space available to compact all existing sstables,
> Cassandra will attempt to perform a partial compaction by removing sstables
> from the set of candidate sstables to be compacted, starting with the largest
> one. It is possible that the sstable removed from the set of sstables to
> compact contains data for which there are tombstones in another (more recent)
> sstable. Since the overlaps between sstables is computed when the
> {{CompactionController}} is created, and the {{CompactionController}} is
> created before the removal of any sstables from the set of sstables to be
> compacted this computed overlap will be outdated when checking which sstables
> are covered by certain tombstones. This leads to the faulty conclusion that
> the tombstones can be pruned during the compaction, causing the data to be
> resurrected.
> The issue is present in Cassandra 4.0 and 4.1. Cassandra 3.11 creates the
> {{CompactionController}} after the set of sstables to compact has been
> reduced, and is thus not affected. {{trunk}} does not appear to support
> partial compactions at all, but instead refuses to compact when the disk is
> full.
> This regression appears to have been introduced by CASSANDRA-13068.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
