[
https://issues.apache.org/jira/browse/CASSANDRA-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225463#comment-13225463
]
Brandon Williams commented on CASSANDRA-4022:
---------------------------------------------
What is happening very reproducibly now is that I started the node, and 5
minutes later the forced compaction check in ACS kicks off, and then I have
looping compaction on the hints but it's only compacting the last sstable over
and over.
> Compaction of hints can get stuck in a loop
> -------------------------------------------
>
> Key: CASSANDRA-4022
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4022
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Priority: Critical
> Fix For: 1.1.0
>
>
> Not exactly sure how I caused this as I was working on something else in
> trunk, but:
> {noformat}
> INFO 17:41:35,682 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-339-Data.db')]
> INFO 17:41:36,430 Compacted to
> [/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-340-Data.db,].
> 4,637,160 to 4,637,160 (~100% of original) bytes
> for 1 keys at 5.912220MB/s. Time: 748ms.
> INFO 17:41:36,431 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-340-Data.db')]
> INFO 17:41:37,238 Compacted to
> [/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-341-Data.db,].
> 4,637,160 to 4,637,160 (~100% of original) bytes
> for 1 keys at 5.479976MB/s. Time: 807ms.
> INFO 17:41:37,239 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-341-Data.db')]
> INFO 17:41:38,163 Compacted to
> [/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-342-Data.db,].
> 4,637,160 to 4,637,160 (~100% of original) bytes
> for 1 keys at 4.786083MB/s. Time: 924ms.
> INFO 17:41:38,164 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-342-Data.db')]
> INFO 17:41:39,014 GC for ParNew: 274 ms for 1 collections, 541261288 used;
> max is 1024458752
> INFO 17:41:39,151 Compacted to
> [/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-343-Data.db,].
> 4,637,160 to 4,637,160 (~100% of original) bytes
> for 1 keys at 4.485132MB/s. Time: 986ms.
> INFO 17:41:39,151 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-343-Data.db')]
> INFO 17:41:40,016 GC for ParNew: 308 ms for 1 collections, 585582200 used;
> max is 1024458752
> INFO 17:41:40,200 Compacted to
> [/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-344-Data.db,].
> 4,637,160 to 4,637,160 (~100% of original) bytes
> for 1 keys at 4.223821MB/s. Time: 1,047ms.
> INFO 17:41:40,201 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-344-Data.db')]
> INFO 17:41:41,017 GC for ParNew: 252 ms for 1 collections, 617877904 used;
> max is 1024458752
> INFO 17:41:41,178 Compacted to
> [/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-345-Data.db,].
> 4,637,160 to 4,637,160 (~100% of original) bytes
> for 1 keys at 4.526449MB/s. Time: 977ms.
> INFO 17:41:41,179 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-345-Data.db')]
> INFO 17:41:41,885 Compacted to
> [/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-346-Data.db,].
> 4,637,160 to 4,637,160 (~100% of original) bytes
> for 1 keys at 6.263938MB/s. Time: 706ms.
> INFO 17:41:41,887 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-346-Data.db')]
> INFO 17:41:42,617 Compacted to
> [/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-347-Data.db,].
> 4,637,160 to 4,637,160 (~100% of original) bytes for 1 keys at
> 6.066311MB/s. Time: 729ms.
> INFO 17:41:42,618 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-347-Data.db')]
> INFO 17:41:43,376 Compacted to
> [/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-348-Data.db,].
> 4,637,160 to 4,637,160 (~100% of original) bytes for 1 keys at
> 5.834222MB/s. Time: 758ms.
> INFO 17:41:43,377 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-348-Data.db')]
> INFO 17:41:44,307 Compacted to
> [/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-349-Data.db,].
> 4,637,160 to 4,637,160 (~100% of original) bytes for 1 keys at
> 4.760323MB/s. Time: 929ms.
> INFO 17:41:44,308 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-349-Data.db')]
> INFO 17:41:45,021 GC for ParNew: 245 ms for 1 collections, 731287832 used;
> max is 1024458752
> INFO 17:41:45,316 Compacted to
> [/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-350-Data.db,].
> 4,637,160 to 4,637,160 (~100% of original) bytes for 1 keys at
> 4.395965MB/s. Time: 1,006ms.
> INFO 17:41:45,316 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-350-Data.db')]
> INFO 17:41:46,022 GC for ParNew: 353 ms for 1 collections, 757476872 used;
> max is 1024458752
> INFO 17:41:46,451 Compacted to
> [/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-351-Data.db,].
> 4,637,160 to 4,637
> {noformat}
> I suspect we broke something subtle in CASSANDRA-3955.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
