[
https://issues.apache.org/jira/browse/CASSANDRA-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108018#comment-13108018
]
Brandon Williams edited comment on CASSANDRA-3224 at 9/19/11 5:43 PM:
----------------------------------------------------------------------
Better, but it looks like LCS starts miscounting at some point, and this causes
compactions to stop:
{noformat}
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,359 LeveledManifest.java (line
218) Level 0 contains 85 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,360 LeveledManifest.java (line
218) Level 1 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,360 LeveledManifest.java (line
218) Level 2 contains 14 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line
218) Level 3 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line
218) Level 4 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line
218) Level 5 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line
218) Level 6 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line
218) Level 7 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line
197) Compaction score for level 0 is 0.0
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,362 LeveledManifest.java (line
197) Compaction score for level 2 is 0.0
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,362
LeveledCompactionStrategy.java (line 112) CompactionManager candidates are
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,362 LeveledManifest.java (line
218) Level 0 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,362 LeveledManifest.java (line
218) Level 1 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,362 LeveledManifest.java (line
218) Level 2 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,362 LeveledManifest.java (line
218) Level 3 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line
218) Level 4 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line
218) Level 5 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line
218) Level 6 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line
218) Level 7 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363
LeveledCompactionStrategy.java (line 112) CompactionManager candidates are
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line
218) Level 0 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line
218) Level 1 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line
218) Level 2 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledManifest.java (line
218) Level 3 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledManifest.java (line
218) Level 4 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledManifest.java (line
218) Level 5 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledManifest.java (line
218) Level 6 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledManifest.java (line
218) Level 7 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364
LeveledCompactionStrategy.java (line 112) CompactionManager candidates are
{noformat}
Actual levels are (according to json manifest):
{noformat}
L0: 85
L1: 0
L2: 14
L3: 0
L4: 0
L5: 0
L6: 0
L7: 0
{noformat}
Attempting to force a compaction via nodetool:
{noformat}
INFO 17:42:28,933 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-2-Data.db'),
SSTableReader(path='/var/lib
/cassandra/data/system/LocationInfo-h-1-Data.db')]
DEBUG 17:42:28,933 Expected bloom filter size : 512
INFO 17:42:28,956 Compacted to
[/var/lib/cassandra/data/system/LocationInfo-h-3-Data.db,]. 498 to 447 (~89%
of original) bytes for 3 key
s at 0.019377MBPS. Time: 22ms.
INFO 17:42:28,956 CF Total Bytes Compacted: 225,632,757
INFO 17:42:28,956 Nothing to compact in Migrations.Use
forceUserDefinedCompaction if you wish to force compaction of single sstables
(e.g. for tombstone collection)
INFO 17:42:28,957 Nothing to compact in Schema.Use forceUserDefinedCompaction
if you wish to force compaction of single sstables (e.g. for tombstone
collection)
{noformat}
was (Author: brandon.williams):
Better, but it looks like LCS starts miscounting at some point, and this
causing compactions to stop:
{noformat}
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,359 LeveledManifest.java (line
218) Level 0 contains 85 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,360 LeveledManifest.java (line
218) Level 1 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,360 LeveledManifest.java (line
218) Level 2 contains 14 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line
218) Level 3 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line
218) Level 4 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line
218) Level 5 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line
218) Level 6 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line
218) Level 7 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line
197) Compaction score for level 0 is 0.0
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,362 LeveledManifest.java (line
197) Compaction score for level 2 is 0.0
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,362
LeveledCompactionStrategy.java (line 112) CompactionManager candidates are
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,362 LeveledManifest.java (line
218) Level 0 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,362 LeveledManifest.java (line
218) Level 1 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,362 LeveledManifest.java (line
218) Level 2 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,362 LeveledManifest.java (line
218) Level 3 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line
218) Level 4 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line
218) Level 5 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line
218) Level 6 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line
218) Level 7 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363
LeveledCompactionStrategy.java (line 112) CompactionManager candidates are
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line
218) Level 0 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line
218) Level 1 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line
218) Level 2 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledManifest.java (line
218) Level 3 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledManifest.java (line
218) Level 4 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledManifest.java (line
218) Level 5 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledManifest.java (line
218) Level 6 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledManifest.java (line
218) Level 7 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364
LeveledCompactionStrategy.java (line 112) CompactionManager candidates are
{noformat}
Actual levels are (according to json manifest):
{noformat}
L0: 85
L1: 0
L2: 14
L3: 0
L4: 0
L5: 0
L6: 0
L7: 0
{noformat}
Attempting to force a compaction via nodetool:
{noformat}
INFO 17:42:28,933 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-2-Data.db'),
SSTableReader(path='/var/lib
/cassandra/data/system/LocationInfo-h-1-Data.db')]
DEBUG 17:42:28,933 Expected bloom filter size : 512
INFO 17:42:28,956 Compacted to
[/var/lib/cassandra/data/system/LocationInfo-h-3-Data.db,]. 498 to 447 (~89%
of original) bytes for 3 key
s at 0.019377MBPS. Time: 22ms.
INFO 17:42:28,956 CF Total Bytes Compacted: 225,632,757
INFO 17:42:28,956 Nothing to compact in Migrations.Use
forceUserDefinedCompaction if you wish to force compaction of single sstables
(e.g. for tombstone collection)
INFO 17:42:28,957 Nothing to compact in Schema.Use forceUserDefinedCompaction
if you wish to force compaction of single sstables (e.g. for tombstone
collection)
{noformat}
> LeveledCompactionStrategy is too complacent
> -------------------------------------------
>
> Key: CASSANDRA-3224
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3224
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Labels: compaction
> Fix For: 1.0.0
>
> Attachments: 3224.txt
>
>
> As the title says, it barely does anything. I inserted 50G worth of data
> with 1G heap and 99% overwrite ratio, and it only compacted twice:
> {noformat}
> INFO [CompactionExecutor:1] 2011-09-16 22:29:54,572 CompactionTask.java
> (line 118) Compacting
> [SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-1-Data.db')]
> INFO [CompactionExecutor:1] 2011-09-16 22:29:58,606 CompactionTask.java
> (line 220) Compacted to
> [/var/lib/cassandra/data/Keyspace1/Standard1-h-2-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-4-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-5-Data.db,].
> 12,595,811 to 12,595,811 (~100% of original) bytes for 40,501 keys at
> 3.058122MBPS. Time: 3,928ms.
> INFO [CompactionExecutor:1] 2011-09-16 22:29:58,607 CompactionTask.java
> (line 222) CF Total Bytes Compacted: 12,595,811
> INFO [CompactionExecutor:3] 2011-09-16 22:29:58,889 CompactionTask.java
> (line 118) Compacting
> [SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-4-Data.db'),
>
> SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-2-Data.db'),
>
> SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-5-Data.db'),
>
> SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-3-Data.db')]
> INFO [CompactionExecutor:3] 2011-09-16 22:30:06,900 CompactionTask.java
> (line 220) Compacted to
> [/var/lib/cassandra/data/Keyspace1/Standard1-h-7-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-9-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-11-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-12-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-14-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-15-Data.db,].
> 28,374,396 to 28,374,396 (~100% of original) bytes for 91,236 keys at
> 3.380379MBPS. Time: 8,005ms.
> INFO [CompactionExecutor:3] 2011-09-16 22:30:06,901 CompactionTask.java
> (line 222) CF Total Bytes Compacted: 40,970,207
> {noformat}
> Resulting in the following levels:
> {noformat}
> L0: 4965
> L1: 6
> L2: 0
> L3: 0
> L4: 0
> L5: 0
> L6: 0
> L7: 0
> {noformat}
> This is obviously going to result in extremely poor read performance.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
