[jira] [Commented] (CASSANDRA-4435) hints compaction loop over same sstable
[
https://issues.apache.org/jira/browse/CASSANDRA-4435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13417275#comment-13417275
]
Jonathan Ellis commented on CASSANDRA-4435:
---
+1
hints compaction loop over same sstable
---
Key: CASSANDRA-4435
URL: https://issues.apache.org/jira/browse/CASSANDRA-4435
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.2
Reporter: Brandon Williams
Assignee: Yuki Morishita
Labels: compaction, hintedhandoff
Attachments: 4435-v2.txt, 4435-v3.txt, 4435.txt
Noticed the following while testing something else:
{noformat}
INFO 22:14:48,496 Completed flushing
/var/lib/cassandra/data/system/hints/system-hints-ia-1-Data.db (109645 bytes)
for commitlog position ReplayPosition(segmentId=9372773011543415,
position=30358488)
INFO 22:14:48,498 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-1-Data.db')]
INFO 22:14:48,500 SSTables for user defined compaction are already being
compacted.
INFO 22:14:48,500 Finished hinted handoff of 16893 rows to endpoint
/10.179.64.227
INFO 22:14:48,658 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-2-Data.db,]. 109,645
to 899 (~0% of original) bytes for 1 keys at 0.005392MB/s. Time: 159ms.
INFO 22:14:48,660 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-2-Data.db')]
INFO 22:14:48,668 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-3-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.107169MB/s. Time: 8ms.
INFO 22:14:48,669 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-3-Data.db')]
INFO 22:14:48,679 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-4-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.095261MB/s. Time: 9ms.
INFO 22:14:48,680 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-4-Data.db')]
INFO 22:14:48,697 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-5-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.050433MB/s. Time: 17ms.
INFO 22:14:48,698 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-5-Data.db')]
INFO 22:14:48,714 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-6-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.053585MB/s. Time: 16ms.
INFO 22:14:48,715 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-6-Data.db')]
INFO 22:14:48,722 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-7-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.122479MB/s. Time: 7ms.
INFO 22:14:48,723 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-7-Data.db')]
INFO 22:14:48,736 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-8-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.065950MB/s. Time: 13ms.
INFO 22:14:48,737 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-8-Data.db')]
INFO 22:14:48,744 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-9-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.122479MB/s. Time: 7ms.
INFO 22:14:48,745 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-9-Data.db')]
INFO 22:14:48,753 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-10-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.122479MB/s. Time: 7ms.
INFO 22:14:48,754 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-10-Data.db')]
INFO 22:14:48,761 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-11-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.122479MB/s. Time: 7ms.
INFO 22:14:48,762 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-11-Data.db')]
INFO 22:14:48,775 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-12-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.065950MB/s. Time: 13ms.
INFO 22:14:48,776 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-12-Data.db')]
INFO 22:14:48,783 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-13-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.122479MB/s. Time: 7ms.
INFO 22:14:48,784 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-13-Data.db')]
INFO
[jira] [Commented] (CASSANDRA-4435) hints compaction loop over same sstable
[
https://issues.apache.org/jira/browse/CASSANDRA-4435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415216#comment-13415216
]
Yuki Morishita commented on CASSANDRA-4435:
---
So far I have not able to reproduce this(If you still have one of those
SSTables, it may help me figure out easier).
But I noticed from the log and the HHOM code, HHOM tries user defined
compaction to compact all SSTables at once after forcing memtable to flush.
Flushing triggers background compaction, so sometimes, it competes with user
defined one. You may notice from the line in above log,
bq. INFO 22:14:48,500 SSTables for user defined compaction are already being
compacted.
shows this situation that SSTables are already marked for compaction when
running user defined.
It may be not the right solution here, but since recursive compaction only
occurs when doing background compaction, I think it is better to add an option
not to submit background compaction after flushing and use it here to make sure
you only run user defined compaction.
hints compaction loop over same sstable
---
Key: CASSANDRA-4435
URL: https://issues.apache.org/jira/browse/CASSANDRA-4435
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.2
Reporter: Brandon Williams
Assignee: Yuki Morishita
Noticed the following while testing something else:
{noformat}
INFO 22:14:48,496 Completed flushing
/var/lib/cassandra/data/system/hints/system-hints-ia-1-Data.db (109645 bytes)
for commitlog position ReplayPosition(segmentId=9372773011543415,
position=30358488)
INFO 22:14:48,498 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-1-Data.db')]
INFO 22:14:48,500 SSTables for user defined compaction are already being
compacted.
INFO 22:14:48,500 Finished hinted handoff of 16893 rows to endpoint
/10.179.64.227
INFO 22:14:48,658 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-2-Data.db,]. 109,645
to 899 (~0% of original) bytes for 1 keys at 0.005392MB/s. Time: 159ms.
INFO 22:14:48,660 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-2-Data.db')]
INFO 22:14:48,668 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-3-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.107169MB/s. Time: 8ms.
INFO 22:14:48,669 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-3-Data.db')]
INFO 22:14:48,679 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-4-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.095261MB/s. Time: 9ms.
INFO 22:14:48,680 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-4-Data.db')]
INFO 22:14:48,697 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-5-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.050433MB/s. Time: 17ms.
INFO 22:14:48,698 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-5-Data.db')]
INFO 22:14:48,714 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-6-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.053585MB/s. Time: 16ms.
INFO 22:14:48,715 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-6-Data.db')]
INFO 22:14:48,722 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-7-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.122479MB/s. Time: 7ms.
INFO 22:14:48,723 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-7-Data.db')]
INFO 22:14:48,736 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-8-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.065950MB/s. Time: 13ms.
INFO 22:14:48,737 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-8-Data.db')]
INFO 22:14:48,744 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-9-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.122479MB/s. Time: 7ms.
INFO 22:14:48,745 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-9-Data.db')]
INFO 22:14:48,753 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-10-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.122479MB/s. Time: 7ms.
INFO 22:14:48,754 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-10-Data.db')]
INFO 22:14:48,761 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-11-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.122479MB/s.
[jira] [Commented] (CASSANDRA-4435) hints compaction loop over same sstable
[
https://issues.apache.org/jira/browse/CASSANDRA-4435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415218#comment-13415218
]
Jonathan Ellis commented on CASSANDRA-4435:
---
That would actually make a lot of sense, since hints are otherwise append-only,
so there's no real benefit from normal background compaction.
hints compaction loop over same sstable
---
Key: CASSANDRA-4435
URL: https://issues.apache.org/jira/browse/CASSANDRA-4435
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.2
Reporter: Brandon Williams
Assignee: Yuki Morishita
Labels: compaction, hintedhandoff
Noticed the following while testing something else:
{noformat}
INFO 22:14:48,496 Completed flushing
/var/lib/cassandra/data/system/hints/system-hints-ia-1-Data.db (109645 bytes)
for commitlog position ReplayPosition(segmentId=9372773011543415,
position=30358488)
INFO 22:14:48,498 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-1-Data.db')]
INFO 22:14:48,500 SSTables for user defined compaction are already being
compacted.
INFO 22:14:48,500 Finished hinted handoff of 16893 rows to endpoint
/10.179.64.227
INFO 22:14:48,658 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-2-Data.db,]. 109,645
to 899 (~0% of original) bytes for 1 keys at 0.005392MB/s. Time: 159ms.
INFO 22:14:48,660 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-2-Data.db')]
INFO 22:14:48,668 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-3-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.107169MB/s. Time: 8ms.
INFO 22:14:48,669 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-3-Data.db')]
INFO 22:14:48,679 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-4-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.095261MB/s. Time: 9ms.
INFO 22:14:48,680 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-4-Data.db')]
INFO 22:14:48,697 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-5-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.050433MB/s. Time: 17ms.
INFO 22:14:48,698 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-5-Data.db')]
INFO 22:14:48,714 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-6-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.053585MB/s. Time: 16ms.
INFO 22:14:48,715 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-6-Data.db')]
INFO 22:14:48,722 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-7-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.122479MB/s. Time: 7ms.
INFO 22:14:48,723 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-7-Data.db')]
INFO 22:14:48,736 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-8-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.065950MB/s. Time: 13ms.
INFO 22:14:48,737 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-8-Data.db')]
INFO 22:14:48,744 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-9-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.122479MB/s. Time: 7ms.
INFO 22:14:48,745 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-9-Data.db')]
INFO 22:14:48,753 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-10-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.122479MB/s. Time: 7ms.
INFO 22:14:48,754 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-10-Data.db')]
INFO 22:14:48,761 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-11-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.122479MB/s. Time: 7ms.
INFO 22:14:48,762 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-11-Data.db')]
INFO 22:14:48,775 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-12-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.065950MB/s. Time: 13ms.
INFO 22:14:48,776 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-12-Data.db')]
INFO 22:14:48,783 Compacted to
[/var/lib/cassandra/data/system/hints/system-hints-ia-13-Data.db,]. 899 to
899 (~100% of original) bytes for 1 keys at 0.122479MB/s. Time: 7ms.
INFO 22:14:48,784 Compacting
