[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196935#comment-14196935 ] Aleksey Yeschenko commented on CASSANDRA-8004: -- I can't shake the feeling that we can do better with a major-er refactor, but I guess not so much in 2.1 - will have to wait for the 3.0 changes. WRT new ACS methods - claiming that all of them have a default implementation, and thus aren't breaking compatibility, is dishonest. All our stock implementations do track their own sstables and thus *have to* override them, and so would any 3rd party compaction strategy implementation. However, given the importance of this change (making incremental repair usable at all), I'm okay with breaking it, so long as the commit makes it to 2.1.2. But please mark ACS#addSSTable() and ACS#removeSSTable() abstract, so that at least nobody gets burned silently. Minor nits: - this reference is redundant in DTCS addSSTable() and removeSSTable() (and while at it, make DTCS.options private and final, and DTCS.sstables final) - same 'this' nit in LCS and STCS Other than that, LGTM/+1 - and if you agree with the comments, just make the modifications on commit. Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.2 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197225#comment-14197225 ] Aleksey Yeschenko commented on CASSANDRA-8004: -- Once last thing. How safe is it for the sstables set to be a simple HashSet, wrt visibility and concurrent updates? Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.2 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191462#comment-14191462 ] Marcus Eriksson commented on CASSANDRA-8004: bq. One issue I see is that this adds new required functions to abstract compactions strategy. Which will break 3rd party compaction strategies. those new methods all have default implementations, and old compaction strategies should still work, even if they don't track their own sstables - they get the sstables to compact from cfs and split them in repaired/unrepaired. We might do it twice per call to 'getNextBackgroundTasks' though, but that should be fine since we mark the sstables as compacting Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.2 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191636#comment-14191636 ] Marcus Eriksson commented on CASSANDRA-8004: CrcCheckChanceTest failed because we deadlocked ourselves due to synchronizing getMaximalTask in WCS - moved the synchronization inside the Callable instead, to avoid that (cfs.runWithCompactionsDisabled calls a few methods in the compaction strategy so moving the synchronization to later avoids the deadlock) Also made DTCS track its sstables rebased and (force) pushed 2 new commits here: https://github.com/krummas/cassandra/commits/marcuse/8004 Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.2 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191826#comment-14191826 ] Jeremiah Jordan commented on CASSANDRA-8004: bq. those new methods all have default implementations, and old compaction strategies should still work, even if they don't track their own sstables - they get the sstables to compact from cfs and split them in repaired/unrepaired. We might do it twice per call to 'getNextBackgroundTasks' though, but that should be fine since we mark the sstables as compacting Hmm, yeah, I guess it should be fine as long as you are using the same strategy for repaired and un-repaired it should probably be OK. The issue I see is if you start doing something like the LCS for repaired STCS for un-repaired. If STCS wasn't tracking its own sstables, then it would trash everything. So if you call 3rd party repair twice, then the 2nd call should just decide nothing to do and move on. Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.2 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191833#comment-14191833 ] Marcus Eriksson commented on CASSANDRA-8004: yeah, for now we always use the same strategy for repaired and unrepaired Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.2 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14190937#comment-14190937 ] Aleksey Yeschenko commented on CASSANDRA-8004: -- Still looking at it. Other than a failing CrcCheckChanceTest, things mostly look good - or rather as good as we can make them in 2.1, without major internal changes (hopefully we'll be able to redesign the APIs surrounding compaction in 3.x). Need a bit more time to make sure it doesn't break anything else though (incl. 3rd party compaction strategies, in unexpected ways). Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.2 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191081#comment-14191081 ] Jeremiah Jordan commented on CASSANDRA-8004: One issue I seenis that this adds new required functions to abstract compactions strategy. Which will break 3rd party compaction strategies. If we are going to be changing the interface, would it be better to have the wrapping strategy track the sstables and pass the list in? Not sure how workable that is. Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.2 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14187639#comment-14187639 ] Jonathan Ellis commented on CASSANDRA-8004: --- (I believe [~iamaleksey] wanted to have a look as well before commit.) Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.2 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177642#comment-14177642 ] sankalp kohli commented on CASSANDRA-8004: -- LGTM +1 Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.2 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173057#comment-14173057 ] sankalp kohli commented on CASSANDRA-8004: -- it would make migrating to incremental repairs so much easier +1. Incremental repair is what I like the most in 2.1 and this is very important for it. Let me review the new patch. Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.2 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163935#comment-14163935 ] sankalp kohli commented on CASSANDRA-8004: -- WrappingCompactionStrategy.startup() again adds stables when they are already done in maybeReloadCompactionStrategy. Also we call startup on both strategies twice. Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.1 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14161824#comment-14161824 ] Marcus Eriksson commented on CASSANDRA-8004: pushed a branch here for this: https://github.com/krummas/cassandra/commit/476b27dc503c3541ee31dacdd70191fee8a819a5 * Introduces a WrappingCompactionStrategy that contains the logic for handling repaired/unrepaired sstables. ** Could be a bit confusing and should probably be refactored for 3.0 - it would be nicer with a CompactionStrategyManager or similar that does not extend AbstractCompactionStrategy, but we currently call cfs.getCompactionStrategy() in many places so having the WCS makes it transparent to any users. * As mentioned in the description this makes it possible, for the first run, to move sstables from the leveling in unrepaired straight over to the repaired-leveling. After the first run, we try to move sstables over, if it fails, they are sent to L0. * keeps 2 instances of the same compaction strategy, changing the compaction strategy is now handled by WrappingCompactionStrategy. * The compaction strategies now track which sstables they can run compaction on (LCS always did this, now STCS does it as well). So the compaction strategy will only ever see either repaired or unrepaired sstables. * As mentioned in CASSANDRA-5351 (and the original reason we did STCS on the unrepaired data) the write amplification gets a lot higher when having 2 parallel levelings, so maybe we should have an option to configure the different compaction strategies separately - you could configure STCS for the unrepaired and LCS for the repaired if the write amplification gets too high for the use case. * An added benefit of running LCS for the unrepaired data is that it makes each sstable contain a smaller range - making it more likely that the sstable is fully contained within the repaired range and the anticompaction step can simply update the repairedAt timestamp and not have to rewrite the entire sstable to split out the repaired ranges. * Also handles the case where someone runs incremental repair once, and then forgets about it, then all the data would be size tiered in the current implementation, with this there will be a small/old repaired leveling and a big unrepaired leveling. Thoughts, comments? Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.1 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14161947#comment-14161947 ] Aleksey Yeschenko commented on CASSANDRA-8004: -- Haven't looked at the patch yet, but the idea sound very good/reasonable to me. Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.1 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14161250#comment-14161250 ] sankalp kohli commented on CASSANDRA-8004: -- Good idea. So if Level 2 stable is repaired, you will find all overlapping stables in L2 repaired and compact them to satisfy the non overlapping constrain? Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.1 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14161494#comment-14161494 ] Marcus Eriksson commented on CASSANDRA-8004: [~kohlisankalp] not initially, what I want to avoid here is the really painful migration to using incremental backups. For the first incremental run, the repaired leveling will be totally empty and we can move sstables over directly. The future incremental repairs will do a can I move this sstable from unrepaired level X to repaired level X?, and the answer will most often be nope, then we drop it to level 0. It might be a good idea to consider doing in the future (3.0) though. I'm targeting 2.1 here and want to keep it as simple as possible. Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Labels: compaction Fix For: 2.1.1 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8004) Run LCS for both repaired and unrepaired data
[ https://issues.apache.org/jira/browse/CASSANDRA-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14147728#comment-14147728 ] T Jake Luciani commented on CASSANDRA-8004: --- +1 Run LCS for both repaired and unrepaired data - Key: CASSANDRA-8004 URL: https://issues.apache.org/jira/browse/CASSANDRA-8004 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Labels: compaction Fix For: 2.1.1 If a user has leveled compaction configured, we should run that for both the unrepaired and the repaired data. I think this would make things a lot easier for end users It would simplify migration to incremental repairs as well, if a user runs incremental repair on its nice leveled unrepaired data, we wont need to drop it all to L0, instead we can just start moving sstables from the unrepaired leveling straight into the repaired leveling Idea could be to have two instances of LeveledCompactionStrategy and move sstables between the instances after an incremental repair run (and let LCS be totally oblivious to whether it handles repaired or unrepaired data). Same should probably apply to any compaction strategy, run two instances and remove all repaired/unrepaired logic from the strategy itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
