[jira] [Updated] (CASSANDRA-3855) RemoveDeleted dominates compaction time for large sstable counts
[ https://issues.apache.org/jira/browse/CASSANDRA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-3855: -- Attachment: 3855.txt So I summarized and attached patch. Tested on trunk and confirmed it fixed. RemoveDeleted dominates compaction time for large sstable counts Key: CASSANDRA-3855 URL: https://issues.apache.org/jira/browse/CASSANDRA-3855 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.0 Reporter: Stu Hood Assignee: Yuki Morishita Labels: compaction, deletes, leveled Attachments: 3855.txt, with-cleaning-java.hprof.txt With very large numbers of sstables (2000+ generated by a `bin/stress -n 100,000,000` run with LeveledCompactionStrategy), PrecompactedRow.removeDeletedAndOldShards dominates compaction runtime, such that commenting it out takes compaction throughput from 200KB/s to 12MB/s. Stack attached. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3855) RemoveDeleted dominates compaction time for large sstable counts
[ https://issues.apache.org/jira/browse/CASSANDRA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3855: -- Assignee: Yuki Morishita (was: Sylvain Lebresne) Yuki, can you benchmark trunk and see if this is still a bottleneck? I think we want to bench both LCS and STCS. RemoveDeleted dominates compaction time for large sstable counts Key: CASSANDRA-3855 URL: https://issues.apache.org/jira/browse/CASSANDRA-3855 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.0 Reporter: Stu Hood Assignee: Yuki Morishita Labels: compaction, deletes, leveled Attachments: with-cleaning-java.hprof.txt With very large numbers of sstables (2000+ generated by a `bin/stress -n 100,000,000` run with LeveledCompactionStrategy), PrecompactedRow.removeDeletedAndOldShards dominates compaction runtime, such that commenting it out takes compaction throughput from 200KB/s to 12MB/s. Stack attached. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3855) RemoveDeleted dominates compaction time for large sstable counts
[ https://issues.apache.org/jira/browse/CASSANDRA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-3855: Attachment: with-cleaning-java.hprof.txt RemoveDeleted dominates compaction time for large sstable counts Key: CASSANDRA-3855 URL: https://issues.apache.org/jira/browse/CASSANDRA-3855 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1 Reporter: Stu Hood Priority: Critical Labels: compaction, deletes, leveled Attachments: with-cleaning-java.hprof.txt With very large numbers of sstables (2000+ generated by a `bin/stress -n 100,000,000` run with LeveledCompactionStrategy), PrecompactedRow.removeDeletedAndOldShards dominates compaction runtime, such that commenting it out takes compaction throughput from 200KB/s to 12MB/s. Stack attached. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
