[jira] [Commented] (CASSANDRA-11119) Add bytes-to-compact estimate
[ https://issues.apache.org/jira/browse/CASSANDRA-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131524#comment-15131524 ] Jonathan Ellis commented on CASSANDRA-9: (Note that STCS-in-L0 doesn't need to be special cased for LCS, since we're just shuffling the work around there, we still ultimately have to move all those bytes into higher levels.) > Add bytes-to-compact estimate > - > > Key: CASSANDRA-9 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Jonathan Ellis > Fix For: 3.x > > > Pending tasks is essentially useless at letting operators know how far behind > compaction is. For STCS, because compaction tasks can vary in size by > several orders of magnitude, so "4 more tasks" is a very wide range. For > LCS, I think the problem is that we only look at first-order work (how much > larger is level X than it "should" be?) and not cascading effects, > particularly in L0 (if we push this into L1, how much does that push into L2 > etc?). > I propose adding an estimate of bytes-to-compact (and deprecating estimated > tasks). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11119) Add bytes-to-compact estimate
[ https://issues.apache.org/jira/browse/CASSANDRA-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131506#comment-15131506 ] Brandon Williams commented on CASSANDRA-9: -- FWIW, I have seen people confused about pending tasks, and ultimately I just end up explaining that it sucks and don't worry about it, so +1. > Add bytes-to-compact estimate > - > > Key: CASSANDRA-9 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Jonathan Ellis > Fix For: 3.x > > > Pending tasks is essentially useless at letting operators know how far behind > compaction is. For STCS, because compaction tasks can vary in size by > several orders of magnitude, so "4 more tasks" is a very wide range. For > LCS, I think the problem is that we only look at first-order work (how much > larger is level X than it "should" be?) and not cascading effects, > particularly in L0 (if we push this into L1, how much does that push into L2 > etc?). > I propose adding an estimate of bytes-to-compact (and deprecating estimated > tasks). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11119) Add bytes-to-compact estimate
[ https://issues.apache.org/jira/browse/CASSANDRA-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131501#comment-15131501 ] Jonathan Ellis commented on CASSANDRA-9: We can actually get pretty accurate with STCS thanks to HyperLogLog. For LCS we're probably still looking at some pretty rough estimates. > Add bytes-to-compact estimate > - > > Key: CASSANDRA-9 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Jonathan Ellis > Fix For: 3.x > > > Pending tasks is essentially useless at letting operators know how far behind > compaction is. For STCS, because compaction tasks can vary in size by > several orders of magnitude, so "4 more tasks" is a very wide range. For > LCS, I think the problem is that we only look at first-order work (how much > larger is level X than it "should" be?) and not cascading effects, > particularly in L0 (if we push this into L1, how much does that push into L2 > etc?). > I propose adding an estimate of bytes-to-compact (and deprecating estimated > tasks). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
