[
https://issues.apache.org/jira/browse/CASSANDRA-3952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223079#comment-13223079
]
Peter Schuller commented on CASSANDRA-3952:
-------------------------------------------
Unable to reproduce even when nuking the .json files and changing back and
fourth between compaction strategies.
However, observe the log below. The last 'Compacting ...' line indicates that
it's compacting a single SStable (first line in log; rest is un-edited log up
to the point of the assertion error).
For the case of a single sstable, the newLevel will be set to 0 first, if the
sstable is in level 0 (which I am assuming, or this makes even less sense, but
is not "known"):
{code}
// avoid increasing the level if we had a single source sstable
involved. This prevents
// cleanup, scrub, and upgradesstables from blowing through the level
cap.
// See CASSANDRA-3989
int newLevel = Iterables.size(removed) == 1
? maximumLevel
: minimumLevel == maximumLevel ? maximumLevel + 1 :
maximumLevel;
{code}
We then re-assign newLevel as such:
{code}
newLevel = skipLevels(newLevel, added);
{code}
But skipLevels() makes no guarantee that it will in fact skip any level:
{code}
private int skipLevels(int newLevel, Iterable<SSTableReader> added)
{
while (maxBytesForLevel(newLevel) < SSTableReader.getTotalBytes(added)
&& generations[(newLevel + 1)].isEmpty())
{
newLevel++;
}
return newLevel;
}
{code}
The other possibility I can see is that the patch somehow introduces a lack of
level information that I'm not seeing, and levelOf() returns -1, and then
skipLevels() skips one level making the final level 0.
Log:
{code}
INFO [CompactionExecutor:4] 2012-03-05 23:29:45,291 CompactionTask.java (line
114) Compacting
[SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-1-Data.db')]
WARN [MutationStage:21] 2012-03-05 23:29:46,181 Memtable.java (line 159)
MemoryMeter uninitialized (jamm not specified as java agent); assuming
liveRatio of 10.0. Usually this means cassandra-env.sh disabled jamm because
you are using a buggy JRE; upgrade to the Sun JRE instead
INFO [FlushWriter:2] 2012-03-05 23:29:46,516 Memtable.java (line 290)
Completed flushing
/var/lib/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-2-Data.db
(471933 bytes)
INFO [OptionalTasks:1] 2012-03-05 23:29:47,178 MeteredFlusher.java (line 58)
flushing high-traffic column family CFS(Keyspace='Keyspace1',
ColumnFamily='Standard1') (estimated 51525937 bytes)
INFO [OptionalTasks:1] 2012-03-05 23:29:47,183 ColumnFamilyStore.java (line
645) Enqueuing flush of Memtable-Standard1@114464782(4122075/51525937
serialized/live bytes, 80825 ops)
INFO [FlushWriter:2] 2012-03-05 23:29:47,184 Memtable.java (line 249) Writing
Memtable-Standard1@114464782(4122075/51525937 serialized/live bytes, 80825 ops)
INFO [FlushWriter:2] 2012-03-05 23:29:48,163 Memtable.java (line 290)
Completed flushing
/var/lib/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-4-Data.db
(583221 bytes)
INFO [OptionalTasks:1] 2012-03-05 23:29:49,186 MeteredFlusher.java (line 58)
flushing high-traffic column family CFS(Keyspace='Keyspace1',
ColumnFamily='Standard1') (estimated 46716000 bytes)
INFO [OptionalTasks:1] 2012-03-05 23:29:49,187 ColumnFamilyStore.java (line
645) Enqueuing flush of Memtable-Standard1@1403886703(3739065/46738312
serialized/live bytes, 73315 ops)
INFO [FlushWriter:2] 2012-03-05 23:29:49,188 Memtable.java (line 249) Writing
Memtable-Standard1@1403886703(3739065/46738312 serialized/live bytes, 73315 ops)
INFO [FlushWriter:2] 2012-03-05 23:29:49,578 Memtable.java (line 290)
Completed flushing
/var/lib/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-5-Data.db
(535826 bytes)
ERROR [CompactionExecutor:4] 2012-03-05 23:29:49,746
AbstractCassandraDaemon.java (line 133) Exception in thread
Thread[CompactionExecutor:4,1,main]
java.lang.AssertionError
{code}
> avoid quadratic startup time in LeveledManifest
> -----------------------------------------------
>
> Key: CASSANDRA-3952
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3952
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Dave Brosius
> Priority: Minor
> Labels: lhf
> Fix For: 1.1.1
>
> Attachments: speed_up_level_of.diff
>
>
> Checking that each sstable is in the manifest on startup is O(N**2) in the
> number of sstables:
> {code}
> . // ensure all SSTables are in the manifest
> for (SSTableReader ssTableReader : cfs.getSSTables())
> {
> if (manifest.levelOf(ssTableReader) < 0)
> manifest.add(ssTableReader);
> }
> {code}
> {code}
> private int levelOf(SSTableReader sstable)
> {
> for (int level = 0; level < generations.length; level++)
> {
> if (generations[level].contains(sstable))
> return level;
> }
> return -1;
> }
> {code}
> Note that the contains call is a linear List.contains.
> We need to switch to a sorted list and bsearch, or a tree, to support
> TB-levels of data in LeveledCompactionStrategy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
