[
https://issues.apache.org/jira/browse/CASSANDRA-12014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502777#comment-15502777
]
Stefania commented on CASSANDRA-12014:
--------------------------------------
Currently running the 3.0 tests for the following patch, based on [~jjirsa]
original code
[here|https://github.com/jeffjirsa/cassandra/commit/f086575751be3e69e4b68ce232e9cb85d222cc04]:
||3.0||trunk||
|[patch|https://github.com/stef1927/cassandra/commits/12014-3.0]|[patch|https://github.com/stef1927/cassandra/commits/12014]|
|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12014-3.0-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12014-testall/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12014-3.0-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12014-dtest/]|
The idea is to avoid hitting the assertion by estimating an average key size
(by looking at either the primary index or the index summaries of existing
sstables), and to automatically increase min_index_interval if the total
estimated size of the index summary is more than 2G.
If we still hit the 2GB limit despite our best efforts to avoid this, we
basically give up on the summary for the final part of the sstable. We could
re-sample on the fly by dropping half the entries or extend the summary with
multiple memory regions but I thought this would be an overkill, and early
opening complicates matters further. I think we can live with a linear scan of
the primary index if we hit 2G in the summary, as far as I understood the code
the only down-side is a linear scan of the primary index for the final part of
the sstable.
> IndexSummary > 2G causes an assertion error
> -------------------------------------------
>
> Key: CASSANDRA-12014
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12014
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Brandon Williams
> Assignee: Stefania
> Priority: Minor
> Fix For: 3.0.x, 3.x
>
>
> {noformat}
> ERROR [CompactionExecutor:1546280] 2016-06-01 13:21:00,444
> CassandraDaemon.java:229 - Exception in thread
> Thread[CompactionExecutor:1546280,1,main]
> java.lang.AssertionError: null
> at
> org.apache.cassandra.io.sstable.IndexSummaryBuilder.maybeAddEntry(IndexSummaryBuilder.java:171)
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at
> org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.append(SSTableWriter.java:634)
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.afterAppend(SSTableWriter.java:179)
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:205)
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:126)
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> ~[na:1.7.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> {noformat}
> I believe this can be fixed by raising the min_index_interval, but we should
> have a better method of coping with this than throwing the AE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
