Ke Han created CASSANDRA-18846:
----------------------------------
Summary: CancellationException when shutting down the 3.11.16 node
Key: CASSANDRA-18846
URL: https://issues.apache.org/jira/browse/CASSANDRA-18846
Project: Cassandra
Issue Type: Bug
Reporter: Ke Han
Attachments: fullSequenceCommands.txt, system-1.log
I encountered the following exception when I tried to shutdown the cassandra
node after drain operation.
{code:java}
INFO [RMI TCP Connection(2)-127.0.0.1] 2023-09-07 12:59:43,440
StorageService.java:1679 - DRAINED
ERROR [MigrationStage:1] 2023-09-07 12:59:43,443 CassandraDaemon.java:244 -
Exception in thread Thread[MigrationStage:1,5,main]
java.util.concurrent.CancellationException: null
at java.util.concurrent.FutureTask.report(FutureTask.java:121)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:438)
at
org.apache.cassandra.db.lifecycle.LogTransaction.waitForDeletions(LogTransaction.java:430)
at
org.apache.cassandra.db.lifecycle.LifecycleTransaction.waitForDeletions(LifecycleTransaction.java:623)
at
org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:587)
at
org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:557)
at org.apache.cassandra.db.Keyspace.unloadCf(Keyspace.java:378)
at org.apache.cassandra.db.Keyspace.dropCf(Keyspace.java:371)
at org.apache.cassandra.config.Schema.dropTable(Schema.java:715)
at
org.apache.cassandra.schema.SchemaKeyspace.lambda$mergeSchema$18(SchemaKeyspace.java:1426)
at java.lang.Iterable.forEach(Iterable.java:75)
at
org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1426)
at
org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1413)
at
org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1390)
at
org.apache.cassandra.service.MigrationManager$1.runMayThrow(MigrationManager.java:464)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84)
at java.lang.Thread.run(Thread.java:750)
ERROR [Native-Transport-Requests-6] 2023-09-07 12:59:43,444
QueryMessage.java:129 - Unexpected error during query
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.util.concurrent.CancellationException
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:442)
at
org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:429)
at
org.apache.cassandra.service.MigrationManager.announceKeyspaceDrop(MigrationManager.java:363)
at
org.apache.cassandra.cql3.statements.DropKeyspaceStatement.announceMigration(DropKeyspaceStatement.java:63)
at
org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:123)
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:243)
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:274)
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:259)
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:116)
at
org.apache.cassandra.transport.Message$Dispatcher.processRequest(Message.java:688)
at
org.apache.cassandra.transport.Message$Dispatcher.lambda$channelRead0$0(Message.java:594)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:113)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.util.concurrent.ExecutionException:
java.util.concurrent.CancellationException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:438)
... 14 common frames omitted
Caused by: java.util.concurrent.CancellationException: null
at java.util.concurrent.FutureTask.report(FutureTask.java:121)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:438)
at
org.apache.cassandra.db.lifecycle.LogTransaction.waitForDeletions(LogTransaction.java:430)
at
org.apache.cassandra.db.lifecycle.LifecycleTransaction.waitForDeletions(LifecycleTransaction.java:623)
at
org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:587)
at
org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:557)
at org.apache.cassandra.db.Keyspace.unloadCf(Keyspace.java:378)
at org.apache.cassandra.db.Keyspace.dropCf(Keyspace.java:371)
at org.apache.cassandra.config.Schema.dropTable(Schema.java:715)
at
org.apache.cassandra.schema.SchemaKeyspace.lambda$mergeSchema$18(SchemaKeyspace.java:1426)
at java.lang.Iterable.forEach(Iterable.java:75)
at
org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1426)
at
org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1413)
at
org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1390)
at
org.apache.cassandra.service.MigrationManager$1.runMayThrow(MigrationManager.java:464)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84)
... 1 common frames omitted
{code}
The CancellationException might be safely handled. I haven't observed data
integrity problem, but I don't know whether it will cause any other effects.
I executed a long list of commands before shutting down the node. It mainly
contains cqlsh related commands. This error message might not be reproduced
deterministically since it's more likely related to interleaving between the
(1) shutdown process and (2) threads working on requests.
I have attached the log file for the complete error message.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
