[ https://issues.apache.org/jira/browse/CASSANDRA-10496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431831#comment-16431831 ]
ASF GitHub Bot commented on CASSANDRA-10496: -------------------------------------------- Github user iksaif commented on the issue: https://github.com/apache/cassandra/pull/147 I likely won't have time to finish, and `unsafe_aggressive_sstable_expiration` is good enough for our usecase now. > Make DTCS/TWCS split partitions based on time during compaction > --------------------------------------------------------------- > > Key: CASSANDRA-10496 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10496 > Project: Cassandra > Issue Type: Improvement > Reporter: Marcus Eriksson > Priority: Major > Labels: dtcs > Fix For: 4.x > > > To avoid getting old data in new time windows with DTCS (or related, like > [TWCS|CASSANDRA-9666]), we need to split out old data into its own sstable > during compaction. > My initial idea is to just create two sstables, when we create the compaction > task we state the start and end times for the window, and any data older than > the window will be put in its own sstable. > By creating a single sstable with old data, we will incrementally get the > windows correct - say we have an sstable with these timestamps: > {{[100, 99, 98, 97, 75, 50, 10]}} > and we are compacting in window {{[100, 80]}} - we would create two sstables: > {{[100, 99, 98, 97]}}, {{[75, 50, 10]}}, and the first window is now > 'correct'. The next compaction would compact in window {{[80, 60]}} and > create sstables {{[75]}}, {{[50, 10]}} etc. > We will probably also want to base the windows on the newest data in the > sstables so that we actually have older data than the window. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org