Mikhail Efremov created IGNITE-27146:
----------------------------------------
Summary: Critical system error detected with Failed to update
distribution zones' logical topology and version keys on
ItTxResourcesVacuumTest#testRecoveryAfterPersistentStateVacuumized
Key: IGNITE-27146
URL: https://issues.apache.org/jira/browse/IGNITE-27146
Project: Ignite
Issue Type: Bug
Reporter: Mikhail Efremov
Attachments: _Integration_Tests_Integration_Transactions_366.log.zip
*Subject test*
{{ItTxResourcesVacuumTest#testRecoveryAfterPersistentStateVacuumized}}
*Reason to fail*
{code:java}
[2025-11-19T00:45:54,257][ERROR][%itrvt_trapsv_3346%Raft-Group-Client-7][FailureManager]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=CRITICAL_ERROR,
failureCtxId=53f58e1c-43ce-42a5-8f5e-c7ae557fc362]
org.apache.ignite.internal.failure.StackTraceCapturingException: Failed to
update distribution zones' logical topology and version keys [topology =
[{id=fa8125d0-c1aa-467c-9f6d-a389d02dff35, name=itrvt_trapsv_3346,
address=127.0.1.1:3346}, {id=71992d35-e120-4efd-bb8e-d12899d195ea,
name=itrvt_trapsv_3344, address=127.0.1.1:3344}], version = 4]
at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:192)
at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:169)
at
org.apache.ignite.internal.distributionzones.DistributionZoneManager.lambda$updateLogicalTopologyInMetaStorage$8(DistributionZoneManager.java:509)
at
java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
at
java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at
java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2090)
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:718)
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:692)
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.lambda$scheduleRetry$51(RaftGroupServiceImpl.java:944)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.util.concurrent.CompletionException:
java.util.concurrent.TimeoutException: Send with retry timed out [retryCount =
150, groupId = metastorage_group, traceId = null, request =
org.apache.ignite.raft.jraft.rpc.WriteActionRequestImpl(org.apache.ignite.internal.metastorage.command.MultiInvokeCommandImpl),
originCommand = null, retryReasons = [[time=1763534749244, msg=Peer
itrvt_trapsv_3346:0 returned code EPERM: Is not leader.;
attemptWaitDuration=193, attemptDuration=7,
attemptStartTime=2025-11-19T00:45:49,244], [time=1763534749445, msg=Peer
itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=193,
attemptDuration=8, attemptStartTime=2025-11-19T00:45:49,445],
[time=1763534749645, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
leader.; attemptWaitDuration=192, attemptDuration=8,
attemptStartTime=2025-11-19T00:45:49,645], [time=1763534749846, msg=Peer
itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=192,
attemptDuration=9, attemptStartTime=2025-11-19T00:45:49,846],
[time=1763534750046, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
leader.; attemptWaitDuration=191, attemptDuration=9,
attemptStartTime=2025-11-19T00:45:50,046], [time=1763534750246, msg=Peer
itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=191,
attemptDuration=9, attemptStartTime=2025-11-19T00:45:50,246],
[time=1763534750447, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
leader.; attemptWaitDuration=201, attemptDuration=0,
attemptStartTime=2025-11-19T00:45:50,447], [time=1763534750647, msg=Peer
itrvt_trapsv_3345:0 threw PeerUnavailableException; attemptWaitDuration=200,
attemptDuration=0, attemptStartTime=2025-11-19T00:45:50,647],
[time=1763534750848, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
leader.; attemptWaitDuration=200, attemptDuration=1,
attemptStartTime=2025-11-19T00:45:50,848], [time=1763534751048, msg=Peer
itrvt_trapsv_3345:0 threw PeerUnavailableException; attemptWaitDuration=199,
attemptDuration=1, attemptStartTime=2025-11-19T00:45:51,048],
[time=1763534751249, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
leader.; attemptWaitDuration=199, attemptDuration=2,
attemptStartTime=2025-11-19T00:45:51,249], [time=1763534751449, msg=Peer
itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=198,
attemptDuration=2, attemptStartTime=2025-11-19T00:45:51,449],
[time=1763534751650, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
leader.; attemptWaitDuration=198, attemptDuration=3,
attemptStartTime=2025-11-19T00:45:51,650], [time=1763534751850, msg=Peer
itrvt_trapsv_3345:0 threw PeerUnavailableException; attemptWaitDuration=197,
attemptDuration=3, attemptStartTime=2025-11-19T00:45:51,850],
[time=1763534752050, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
leader.; attemptWaitDuration=197, attemptDuration=3,
attemptStartTime=2025-11-19T00:45:52,050], [time=1763534752251, msg=Peer
itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=197,
attemptDuration=4, attemptStartTime=2025-11-19T00:45:52,251],
[time=1763534752451, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
leader.; attemptWaitDuration=196, attemptDuration=4,
attemptStartTime=2025-11-19T00:45:52,451], [time=1763534752651, msg=Peer
itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=196,
attemptDuration=4, attemptStartTime=2025-11-19T00:45:52,651],
[time=1763534752852, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
leader.; attemptWaitDuration=196, attemptDuration=5,
attemptStartTime=2025-11-19T00:45:52,852], [time=1763534753052, msg=Peer
itrvt_trapsv_3345:0 threw PeerUnavailableException; attemptWaitDuration=195,
attemptDuration=5, attemptStartTime=2025-11-19T00:45:53,052],
[time=1763534753252, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
leader.; attemptWaitDuration=195, attemptDuration=5,
attemptStartTime=2025-11-19T00:45:53,252], [time=1763534753452, msg=Peer
itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=195,
attemptDuration=5, attemptStartTime=2025-11-19T00:45:53,452],
[time=1763534753653, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
leader.; attemptWaitDuration=195, attemptDuration=6,
attemptStartTime=2025-11-19T00:45:53,653], [time=1763534753853, msg=Peer
itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=194,
attemptDuration=6, attemptStartTime=2025-11-19T00:45:53,853],
[time=1763534754054, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
leader.; attemptWaitDuration=194, attemptDuration=7,
attemptStartTime=2025-11-19T00:45:54,054]], stopTime = 1763534754193,
currentTime = 1763534754254, startTime = 1763534724194, duration = 30060].
{code}
*Failed TC build*
[link|https://ci.ignite.apache.org/buildConfiguration/ApacheIgnite3x_Test_RunAllTests/9661272]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
