[ 
https://issues.apache.org/jira/browse/CASSANDRA-12778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586793#comment-15586793
 ] 

Sharvanath Pathak commented on CASSANDRA-12778:
-----------------------------------------------

There are multiple problems right now:
i) There can be cases when the tombstones can be looping in the cluster 
forever. For example, even after a node purges a tombstones it can receive the 
same from read-repair and can rewrite this old tombstones as unrepaired. We are 
seeing this happen even with read-repair disabled and are debugging what else 
could cause this.

iI) Flush almost always triggers a compaction. This causes the data in these 
old SSTables to not be repaired because the old SSTable which has been repaired 
is gone when it comes to marking them repaired. Seems like this is a race and 
compaction should be disallowed (on some fine granularity, e.g. keyspace) when 
repair is running.

iiI) The check for whether there is any overlapping data in other SSTables is 
very coarse in granularity in that it looks at the minimumTimestamp of the 
non-compacting SSTables.


> Tombstones not being deleted when only_purge_repaired_tombstones is enabled
> ---------------------------------------------------------------------------
>
>                 Key: CASSANDRA-12778
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12778
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Arvind Nithrakashyap
>            Assignee: Marcus Eriksson
>            Priority: Critical
>
> When we use only_purge_repaired_tombstones for compaction, we noticed that 
> tombstones are no longer being deleted.
> {noformat}compaction = {'class': 
> 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy', 
> 'only_purge_repaired_tombstones': 'true'}{noformat}
> The root cause for this seems to be caused by the fact that repair itself 
> issues a flush which in turn leads to a new sstable being created (which is 
> not in the repair set). It looks like we do have some old data in this 
> sstable because of this, only tombstones older than that timestamp are 
> getting deleted even though many more keys have been repaired. 
> Fundamentally it looks like flush and repair can race with each other and 
> with leveled compaction, the flush creates a new sstable at level 0 and 
> removes the older sstable (the one that is picked for repair). Since repair 
> itself seems to issue multiple flushes, the level 0 sstable never gets 
> repaired and hence tombstones never get deleted. 
> We have already included the fix for CASSANDRA-12703 while testing. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to