[
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144901#comment-16144901
]
Romain GERARD edited comment on CASSANDRA-13418 at 8/29/17 8:10 AM:
--------------------------------------------------------------------
bq. 2. only enabling unsafe_aggressive_sstable_expiration
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *not ignore* the overlaps and
look globally if the current sstable is eligible.
bq. 1. enabling both
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *ignore* the overlaps and look
globally if the current sstable is eligible.
-----
I made a new version of the patch with uncheckedTombstoneCompaction disabled
and a warning message.
https://github.com/criteo-forks/cassandra/commit/1800b23ddfbb308645c44022e15c1760a0124025
the diff
{noformat}
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index 43c90c7042..d21222c484 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
else
logger.debug("Enabling tombstone compactions for TWCS");
- if (this.options.ignoreOverlaps)
- this.uncheckedTombstoneCompaction = true;
-
+ if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) {
+ logger.warn("You are running with sstables overlapping checks
disabled but without unchecked tombstone compaction, check that this is what
you want");
+ }
}
{noformat}
was (Author: rgerard):
bq. 2. only enabling unsafe_aggressive_sstable_expiration
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *not ignore* the overlaps and
look globally if the current sstable is eligible.
bq. 1. enabling both
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *ignore* the overlaps and look
globally if the current sstable is eligible.
-----
I made a new version of the patch with uncheckedTombstoneCompaction disabled
and a warning message.
https://github.com/criteo-forks/cassandra/commit/800ab325cbf7d9d4d5e60e2b959918426e121815
the diff
{noformat}
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index 43c90c7042..d21222c484 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
else
logger.debug("Enabling tombstone compactions for TWCS");
- if (this.options.ignoreOverlaps)
- this.uncheckedTombstoneCompaction = true;
-
+ if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) {
+ logger.warn("You are running with sstables overlapping checks
disabled but without unchecked tombstone compaction, check that this what you
want");
+ }
}
{noformat}
> Allow TWCS to ignore overlaps when dropping fully expired sstables
> ------------------------------------------------------------------
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
> Issue Type: Improvement
> Components: Compaction
> Reporter: Corentin Chary
> Labels: twcs
> Attachments: twcs-cleanup.png
>
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If
> you really want read-repairs you're going to have sstables blocking the
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a
> very low value and that will purge the blockers of old data that should
> already have expired, thus removing the overlaps and allowing the other
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have
> time series, you might not care if all your data doesn't exactly expire at
> the right time, or if data re-appears for some time, as long as it gets
> deleted as soon as it can. And in this situation I believe it would be really
> beneficial to allow users to simply ignore overlapping SSTables when looking
> for fully expired ones.
> To the question: why would you need read-repairs ?
> - Full repairs basically take longer than the TTL of the data on my dataset,
> so this isn't really effective.
> - Even with a 10% chances of doing a repair, we found out that this would be
> enough to greatly reduce entropy of the most used data (and if you have
> timeseries, you're likely to have a dashboard doing the same important
> queries over and over again).
> - LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow.
> I'll try to come up with a patch demonstrating how this would work, try it on
> our system and report the effects.
> cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
