[
https://issues.apache.org/jira/browse/CASSANDRA-11127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15246302#comment-15246302
]
Jim Witschey commented on CASSANDRA-11127:
------------------------------------------
Saw another failure, this time on 2.1:
http://cassci.datastax.com/job/cassandra-2.1_dtest_jdk8/209/testReport/index_summary_upgrade_test/TestUpgradeIndexSummary/test_upgrade_index_summary/
{code}
Error Message
Unexpected error in log, see stdout
-------------------- >> begin captured logging << --------------------
dtest: DEBUG: cluster ccm directory: /mnt/tmp/dtest-cgBC2o
dtest: DEBUG: Custom init_config not found. Setting defaults.
dtest: DEBUG: Done setting configuration options:
{ 'initial_token': None,
'num_tokens': '32',
'phi_convict_threshold': 5,
'range_request_timeout_in_ms': 10000,
'read_request_timeout_in_ms': 10000,
'request_timeout_in_ms': 10000,
'truncate_request_timeout_in_ms': 10000,
'write_request_timeout_in_ms': 10000}
dtest: DEBUG: Set new cassandra dir for node1:
/home/automaton/.ccm/repository/2.1.3
dtest: DEBUG: Set new cassandra dir for node1: /home/automaton/cassandra
dtest: DEBUG: removing ccm cluster test at: /mnt/tmp/dtest-cgBC2o
dtest: DEBUG: clearing ssl stores from [/mnt/tmp/dtest-cgBC2o] directory
--------------------- >> end captured logging << ---------------------
Stacktrace
File "/usr/lib/python2.7/unittest/case.py", line 358, in run
self.tearDown()
File "/home/automaton/cassandra-dtest/dtest.py", line 693, in tearDown
raise AssertionError('Unexpected error in log, see stdout')
"Unexpected error in log, see stdout\n-------------------- >> begin captured
logging << --------------------\ndtest: DEBUG: cluster ccm directory:
/mnt/tmp/dtest-cgBC2o\ndtest: DEBUG: Custom init_config not found. Setting
defaults.\ndtest: DEBUG: Done setting configuration options:\n{
'initial_token': None,\n 'num_tokens': '32',\n 'phi_convict_threshold':
5,\n 'range_request_timeout_in_ms': 10000,\n
'read_request_timeout_in_ms': 10000,\n 'request_timeout_in_ms': 10000,\n
'truncate_request_timeout_in_ms': 10000,\n 'write_request_timeout_in_ms':
10000}\ndtest: DEBUG: Set new cassandra dir for node1:
/home/automaton/.ccm/repository/2.1.3\ndtest: DEBUG: Set new cassandra dir for
node1: /home/automaton/cassandra\ndtest: DEBUG: removing ccm cluster test at:
/mnt/tmp/dtest-cgBC2o\ndtest: DEBUG: clearing ssl stores from
[/mnt/tmp/dtest-cgBC2o] directory\n--------------------- >> end captured
logging << ---------------------"
Standard Output
Unexpected error in node1 log, error:
ERROR [CompactionExecutor:1] 2016-04-15 16:17:54,181 CassandraDaemon.java:167 -
Exception in thread Thread[CompactionExecutor:1,1,main]
java.util.concurrent.RejectedExecutionException: Task
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@4895f342
rejected from
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor@5e9d7674[Terminated,
pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 10]
at
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
~[na:1.8.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:326)
~[na:1.8.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
~[na:1.8.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:622)
~[na:1.8.0_45]
at
org.apache.cassandra.io.sstable.SSTableReader$Tidier.scheduleTidy(SSTableReader.java:2041)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.io.sstable.SSTableReader$Tidier.tidy(SSTableReader.java:2025)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.utils.concurrent.RefCountedImpl$GlobalState.release(RefCountedImpl.java:86)
~[apache-cassandra-2.1.3.jar:2.1.3]
at org.apache.cassandra.utils.concurrent.Ref$State.release(Ref.java:77)
~[apache-cassandra-2.1.3.jar:2.1.3]
at org.apache.cassandra.utils.concurrent.Ref.release(Ref.java:32)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.db.DataTracker.replaceReaders(DataTracker.java:409)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.db.DataTracker.replaceEarlyOpenedFiles(DataTracker.java:318)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.io.sstable.SSTableRewriter.replaceWithFinishedReaders(SSTableRewriter.java:390)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:372)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:329)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:309)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:213)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:76)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:240)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
Suppressed: java.util.concurrent.RejectedExecutionException: Task
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@3ed58177
rejected from
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor@5e9d7674[Terminated,
pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 10]
at
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
~[na:1.8.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:326)
~[na:1.8.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
~[na:1.8.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:622)
~[na:1.8.0_45]
at
org.apache.cassandra.io.sstable.SSTableReader$Tidier.scheduleTidy(SSTableReader.java:2041)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.io.sstable.SSTableReader$Tidier.tidy(SSTableReader.java:2025)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.utils.concurrent.RefCountedImpl$GlobalState.release(RefCountedImpl.java:86)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.utils.concurrent.Ref$State.release(Ref.java:77)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.utils.concurrent.Ref.release(Ref.java:32)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.db.DataTracker.replaceReaders(DataTracker.java:409)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.db.DataTracker.replaceWithNewInstances(DataTracker.java:303)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.io.sstable.SSTableRewriter.moveStarts(SSTableRewriter.java:254)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.io.sstable.SSTableRewriter.abort(SSTableRewriter.java:180)
~[apache-cassandra-2.1.3.jar:2.1.3]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:219)
~[apache-cassandra-2.1.3.jar:2.1.3]
... 9 common frames omitted
{code}
> index_summary_upgrade_test.py is failing
> ----------------------------------------
>
> Key: CASSANDRA-11127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11127
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
> Reporter: Philip Thompson
> Assignee: DS Test Eng
> Labels: dtest, windows
> Fix For: 3.0.x
>
> Attachments: node1_debug.log
>
>
> index_summary_upgrade_test.py is failing on the cassandra-3.0 branch, when
> run without vnodes. The exception I'm seeing on cassci is different than
> locally. The cassci failure is
> [here|http://cassci.datastax.com/job/cassandra-3.0_novnode_dtest/157/testReport/index_summary_upgrade_test/TestUpgradeIndexSummary/test_upgrade_index_summary/].
> Locally I see the following:
> {code}
> 'ERROR [SSTableBatchOpen:2] 2016-02-05 15:29:04,304 CassandraDaemon.java:195
> - Exception in thread
> Thread[SSTableBatchOpen:2,5,main]\njava.lang.AssertionError: Illegal bounds
> [4..8); size: 4\n\tat
> org.apache.cassandra.io.util.Memory.checkBounds(Memory.java:339)
> ~[main/:na]\n\tat org.apache.cassandra.io.util.Memory.getInt(Memory.java:292)
> ~[main/:na]\n\tat
> org.apache.cassandra.io.sstable.IndexSummary.getPositionInSummary(IndexSummary.java:146)
> ~[main/:na]\n\tat
> org.apache.cassandra.io.sstable.IndexSummary.getKey(IndexSummary.java:151)
> ~[main/:na]\n\tat
> org.apache.cassandra.io.sstable.format.SSTableReader.validateSummarySamplingLevel(SSTableReader.java:928)
> ~[main/:na]\n\tat
> org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:748)
> ~[main/:na]\n\tat
> org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:705)
> ~[main/:na]\n\tat
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:491)
> ~[main/:na]\n\tat
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:374)
> ~[main/:na]\n\tat
> org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:533)
> ~[main/:na]\n\tat
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_66]\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_66]\n\tat
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_66]\n\tat
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_66]\n\tat java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]']
> {code}
> Node log is attached.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
