[
https://issues.apache.org/jira/browse/CASSANDRA-12364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15404194#comment-15404194
]
Sylvain Lebresne commented on CASSANDRA-12364:
----------------------------------------------
The stack says that an internal schema propagation message raced with a node
shutting down. We can {{ColumnFamilyStore.waitForFlushes}} to catch the
exception, check we do are shutting down, and ignore the problem otherwise, and
we probably should, but it's not the first time we see that kind of exception
and it's never more than an annoyance (we're only ever shutting down pools on
shutdown, as we should, and Cassandra being fault-tolerant, it's never a
problem that something else race with such shutdown), so I wonder if the test
framework shouldn't do an exception for that specific exception (it can always
report it on the standard output, but provided that happens while shutting down
a node (or during test teardown() in particular), it's probably ok to ignore
it).
> dtest failure in
> upgrade_tests.paging_test.TestPagingDatasetChangesNodes3RF3_Upgrade_current_3_x_To_indev_3_x.test_cell_TTL_expiry_during_paging
> ------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-12364
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12364
> Project: Cassandra
> Issue Type: Test
> Reporter: Sean McCarthy
> Assignee: DS Test Eng
> Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log,
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest_upgrade/10/testReport/upgrade_tests.paging_test/TestPagingDatasetChangesNodes3RF3_Upgrade_current_3_x_To_indev_3_x/test_cell_TTL_expiry_during_paging
> {code}
> Error Message
> Unexpected error in log, see stdout
> {code}
> {code}
> Standard Output
> http://git-wip-us.apache.org/repos/asf/cassandra.git
> git:5051c0f6eb3f984600600c9577d6b5ece9038c74
> Unexpected error in node1 log, error:
> ERROR [InternalResponseStage:4] 2016-07-28 03:23:02,097
> CassandraDaemon.java:217 - Exception in thread
> Thread[InternalResponseStage:4,5,main]
> java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut
> down
> at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:61)
> ~[apache-cassandra-3.7.jar:3.7]
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
> ~[na:1.8.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
> ~[na:1.8.0_51]
> at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:165)
> ~[apache-cassandra-3.7.jar:3.7]
> at
> org.apache.cassandra.db.ColumnFamilyStore.waitForFlushes(ColumnFamilyStore.java:930)
> ~[apache-cassandra-3.7.jar:3.7]
> at
> org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:892)
> ~[apache-cassandra-3.7.jar:3.7]
> at
> org.apache.cassandra.schema.SchemaKeyspace.lambda$flush$1(SchemaKeyspace.java:279)
> ~[apache-cassandra-3.7.jar:3.7]
> at
> org.apache.cassandra.schema.SchemaKeyspace$$Lambda$200/2113926365.accept(Unknown
> Source) ~[na:na]
> at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_51]
> at
> org.apache.cassandra.schema.SchemaKeyspace.flush(SchemaKeyspace.java:279)
> ~[apache-cassandra-3.7.jar:3.7]
> at
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1271)
> ~[apache-cassandra-3.7.jar:3.7]
> at
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1253)
> ~[apache-cassandra-3.7.jar:3.7]
> at
> org.apache.cassandra.service.MigrationTask$1.response(MigrationTask.java:92)
> ~[apache-cassandra-3.7.jar:3.7]
> at
> org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:53)
> ~[apache-cassandra-3.7.jar:3.7]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64)
> ~[apache-cassandra-3.7.jar:3.7]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_51]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
> {code}
> Related failures:
> http://cassci.datastax.com/job/trunk_dtest_upgrade/11/testReport/upgrade_tests.paging_test/TestPagingWithDeletionsNodes3RF3_Upgrade_current_3_x_To_indev_3_x/test_ttl_deletions/
> http://cassci.datastax.com/job/trunk_dtest_upgrade/12/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3_Upgrade_current_3_x_To_indev_3_x/static_columns_with_distinct_test/
> http://cassci.datastax.com/job/trunk_dtest_upgrade/12/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3_Upgrade_current_3_x_To_indev_3_x/refuse_in_with_indexes_test/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
