[
https://issues.apache.org/jira/browse/IGNITE-25295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Puchkovskiy updated IGNITE-25295:
---------------------------------------
Description:
[2025-05-05T08:23:28,513][ERROR][%idft_tcpoc_3346%partition-operations-5][FailureManager]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=CRITICAL_ERROR]
org.apache.ignite.internal.failure.StackTraceCapturingException: Failed to scan
transaction state storage [commitPartition=21_part_0].
at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:153)
~[ignite-failure-handler-9.0.127-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:130)
~[ignite-failure-handler-9.0.127-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.partition.replicator.handlers.TxCleanupRecoveryRequestHandler.runPersistentStorageScan(TxCleanupRecoveryRequestHandler.java:100)
~[ignite-partition-replicator-9.0.127-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.partition.replicator.handlers.TxCleanupRecoveryRequestHandler.handle(TxCleanupRecoveryRequestHandler.java:62)
~[ignite-partition-replicator-9.0.127-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.processRequest(PartitionReplicaListener.java:597)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.processRequestInContext(PartitionReplicaListener.java:543)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.lambda$invoke$0(PartitionReplicaListener.java:530)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
at
java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187)
~[?:?]
at
java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2309)
~[?:?]
at
org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.invoke(PartitionReplicaListener.java:530)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.replicator.ReplicaImpl.processRequest(ReplicaImpl.java:124)
~[ignite-replicator-9.0.127-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.replicator.ReplicaManager.handleReplicaRequest(ReplicaManager.java:395)
~[ignite-replicator-9.0.127-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.replicator.ReplicaManager.lambda$onReplicaMessageReceived$0(ReplicaManager.java:326)
~[ignite-replicator-9.0.127-SNAPSHOT.jar:?]
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[?:?]
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[?:?]
at java.base/java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: org.apache.ignite.internal.lang.IgniteInternalException: Transaction
state storage is stopped: [table=21, partitionId=0]
at
org.apache.ignite.internal.tx.storage.state.rocksdb.TxStateRocksDbPartitionStorage.throwExceptionDependingOnStorageState(TxStateRocksDbPartitionStorage.java:631)
~[ignite-transactions-9.0.127-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.tx.storage.state.rocksdb.TxStateRocksDbPartitionStorage.busy(TxStateRocksDbPartitionStorage.java:659)
~[ignite-transactions-9.0.127-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.tx.storage.state.rocksdb.TxStateRocksDbPartitionStorage$1.hasNext(TxStateRocksDbPartitionStorage.java:314)
~[ignite-transactions-9.0.127-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.worker.ThreadAssertingCursor.hasNext(ThreadAssertingCursor.java:39)
~[ignite-workers-9.0.127-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.partition.replicator.handlers.TxCleanupRecoveryRequestHandler.runPersistentStorageScan(TxCleanupRecoveryRequestHandler.java:72)
~[ignite-partition-replicator-9.0.127-SNAPSHOT.jar:?]
... 13 more
> Protect PartitionReplicaListener with busy lock
> -----------------------------------------------
>
> Key: IGNITE-25295
> URL: https://issues.apache.org/jira/browse/IGNITE-25295
> Project: Ignite
> Issue Type: Improvement
> Reporter: Roman Puchkovskiy
> Assignee: Roman Puchkovskiy
> Priority: Major
> Labels: ignite-3
>
> [2025-05-05T08:23:28,513][ERROR][%idft_tcpoc_3346%partition-operations-5][FailureManager]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=CRITICAL_ERROR]
> org.apache.ignite.internal.failure.StackTraceCapturingException: Failed to
> scan transaction state storage [commitPartition=21_part_0].
> at
> org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:153)
> ~[ignite-failure-handler-9.0.127-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:130)
> ~[ignite-failure-handler-9.0.127-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.partition.replicator.handlers.TxCleanupRecoveryRequestHandler.runPersistentStorageScan(TxCleanupRecoveryRequestHandler.java:100)
> ~[ignite-partition-replicator-9.0.127-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.partition.replicator.handlers.TxCleanupRecoveryRequestHandler.handle(TxCleanupRecoveryRequestHandler.java:62)
> ~[ignite-partition-replicator-9.0.127-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.processRequest(PartitionReplicaListener.java:597)
> ~[ignite-table-9.0.127-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.processRequestInContext(PartitionReplicaListener.java:543)
> ~[ignite-table-9.0.127-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.lambda$invoke$0(PartitionReplicaListener.java:530)
> ~[ignite-table-9.0.127-SNAPSHOT.jar:?]
> at
> java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187)
> ~[?:?]
> at
> java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2309)
> ~[?:?]
> at
> org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.invoke(PartitionReplicaListener.java:530)
> ~[ignite-table-9.0.127-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.replicator.ReplicaImpl.processRequest(ReplicaImpl.java:124)
> ~[ignite-replicator-9.0.127-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.replicator.ReplicaManager.handleReplicaRequest(ReplicaManager.java:395)
> ~[ignite-replicator-9.0.127-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.replicator.ReplicaManager.lambda$onReplicaMessageReceived$0(ReplicaManager.java:326)
> ~[ignite-replicator-9.0.127-SNAPSHOT.jar:?]
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> [?:?]
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> [?:?]
> at java.base/java.lang.Thread.run(Thread.java:833) [?:?]
> Caused by: org.apache.ignite.internal.lang.IgniteInternalException:
> Transaction state storage is stopped: [table=21, partitionId=0]
> at
> org.apache.ignite.internal.tx.storage.state.rocksdb.TxStateRocksDbPartitionStorage.throwExceptionDependingOnStorageState(TxStateRocksDbPartitionStorage.java:631)
> ~[ignite-transactions-9.0.127-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.tx.storage.state.rocksdb.TxStateRocksDbPartitionStorage.busy(TxStateRocksDbPartitionStorage.java:659)
> ~[ignite-transactions-9.0.127-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.tx.storage.state.rocksdb.TxStateRocksDbPartitionStorage$1.hasNext(TxStateRocksDbPartitionStorage.java:314)
> ~[ignite-transactions-9.0.127-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.worker.ThreadAssertingCursor.hasNext(ThreadAssertingCursor.java:39)
> ~[ignite-workers-9.0.127-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.partition.replicator.handlers.TxCleanupRecoveryRequestHandler.runPersistentStorageScan(TxCleanupRecoveryRequestHandler.java:72)
> ~[ignite-partition-replicator-9.0.127-SNAPSHOT.jar:?]
> ... 13 more
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
