[jira] [Commented] (CASSANDRA-8406) Add option to set max_sstable_age in seconds in DTCS
[ https://issues.apache.org/jira/browse/CASSANDRA-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14308037#comment-14308037 ] Björn Hegerfors commented on CASSANDRA-8406: [~krummas] well we did try max_sstable_age_days=1 in a new table, just the other day. This table has a default_time_to_live at 1 day, and the hope was that SSTables older than that would just disappear. That's not the case, and it may be timely for me to address this ticket now: CASSANDRA-8359. With that fixed, I think that having max_sstable_age equal to the default_time_to_live of the table is the obviously right setting, giving very efficient behavior. Why compact something that's about to go away? If it works as well as I hope, the next logical step might be to set max_sstable_age automatically when default_time_to_live is set. And if someone wants to use default_time_to_live 1 day, I don't see why they shouldn't be able to use max_sstable_age 1 day. So I do see a case for max_sstable_age being less than days. And for something like this, having the setting in seconds seems most appropriate. But the floating point solution is probably a viable compromise. I suppose the main argument for having the setting in seconds is: why is default_time_to_live in seconds when max_sstable_age is not? Add option to set max_sstable_age in seconds in DTCS Key: CASSANDRA-8406 URL: https://issues.apache.org/jira/browse/CASSANDRA-8406 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.0.13 Attachments: 0001-8406.patch, 0001-patch.patch Using days as the unit for max_sstable_age in DTCS might be too much, add option to set it in seconds -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8406) Add option to set max_sstable_age in seconds in DTCS
[ https://issues.apache.org/jira/browse/CASSANDRA-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306403#comment-14306403 ] Jonathan Shook commented on CASSANDRA-8406: --- +1 on 0001-8406.patch Add option to set max_sstable_age in seconds in DTCS Key: CASSANDRA-8406 URL: https://issues.apache.org/jira/browse/CASSANDRA-8406 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.0.13 Attachments: 0001-8406.patch, 0001-patch.patch Using days as the unit for max_sstable_age in DTCS might be too much, add option to set it in seconds -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8406) Add option to set max_sstable_age in seconds in DTCS
[ https://issues.apache.org/jira/browse/CASSANDRA-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14241369#comment-14241369 ] Björn Hegerfors commented on CASSANDRA-8406: I proposed another approach in my last comment on CASSANDRA-8340 (which, by the way, is very tightly coupled to this ticket). The idea is to specify max window re-merge instead of max sstable age. That option would mean, very nearly, how many times do you want each value to be rewritten?. The good thing about that option in this context is that it scales relatively to window size. If small time windows are used (low baseTime), then a small max_window_exponent will indeed lead to a max SSTable age far lower than a day. Consider min_threshold=4 and base_time_seconds=60. Then max_window_exponent=3 would create all the way up to 64-minute windows, and stop after that. With max_window_exponent=10, the largest windows will be ~2 years (actually ~1.995 years, coincidentally). I can implement this. It would not be difficult. But what do you think? Is this option too confusing? Is it a bad thing that changing base_time_seconds also changes the max SSTable age (linearly)? And that min_threshold does the same (polynomially)? It's just that the number of recompactions is what this is all about anyway. So why not be explicit about it? On a second note, would it make sense for some other behavior than no more compactions ever after SSTables get too old? For instance, how about a flag that makes DTCS create infinitely many same-size windows preceding the max window size? So in my first example, infinite 64-minute windows would be produced. In the event of a repair or out-of-order write, a window many days old may be touched and a compaction would trigger in that window. I'm not suggesting this as a default, but maybe it's useful for something? Add option to set max_sstable_age in seconds in DTCS Key: CASSANDRA-8406 URL: https://issues.apache.org/jira/browse/CASSANDRA-8406 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.0.12 Attachments: 0001-patch.patch Using days as the unit for max_sstable_age in DTCS might be too much, add option to set it in seconds -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8406) Add option to set max_sstable_age in seconds in DTCS
[ https://issues.apache.org/jira/browse/CASSANDRA-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14236478#comment-14236478 ] Jonathan Ellis commented on CASSANDRA-8406: --- I'm still -0 on adding two options for the same functionality. And I still think days is usually the right unit to encourage in production. Add option to set max_sstable_age in seconds in DTCS Key: CASSANDRA-8406 URL: https://issues.apache.org/jira/browse/CASSANDRA-8406 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.0.12 Attachments: 0001-patch.patch Using days as the unit for max_sstable_age in DTCS might be too much, add option to set it in seconds -- This message was sent by Atlassian JIRA (v6.3.4#6332)
