amaliujia commented on PR #11032: URL: https://github.com/apache/ozone/pull/11032#issuecomment-5322682347
> `org.apache.hadoop.ozone.om.TestOzoneManagerHAWithStoppedNodes` might also be a real test failure. > > ``` > [ERROR] org.apache.hadoop.ozone.om.TestOzoneManagerHAWithStoppedNodes.testRetryCacheWithDownedOM -- Time elapsed: 15.10 s <<< ERROR! > org.apache.ratis.protocol.exceptions.TransferLeadershipException: omNode-3@group-523986131536: Failed to transfer leadership to omNode-1 (the current leader is omNode-3): TIMED_OUT(10s) > at org.apache.ratis.server.impl.TransferLeadership$PendingRequest.complete(TransferLeadership.java:152) > at org.apache.ratis.server.impl.TransferLeadership.lambda$complete$14(TransferLeadership.java:345) > at java.base/java.util.Optional.ifPresent(Optional.java:178) > at org.apache.ratis.server.impl.TransferLeadership.complete(TransferLeadership.java:345) > at org.apache.ratis.server.impl.TransferLeadership.lambda$start$9(TransferLeadership.java:310) > at org.apache.ratis.util.TimeoutTimer.lambda$onTimeout$2(TimeoutTimer.java:101) > at org.apache.ratis.util.LogUtils.runAndLog(LogUtils.java:38) > at org.apache.ratis.util.LogUtils$1.run(LogUtils.java:78) > at org.apache.ratis.util.TimeoutTimer$Task.run(TimeoutTimer.java:55) > at java.base/java.util.TimerThread.mainLoop(Timer.java:572) > at java.base/java.util.TimerThread.run(Timer.java:522) > ``` > > even if I increase the timeout to `100 * 1000` as 100 seconds, the leader transfer can still not finish and timeout: > > https://github.com/apache/ozone/blob/548818786814d9c7ad3cfe5689afdafd74b67a6d/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerHAWithStoppedNodes.java#L698 Fix `TestOzoneManagerHAWithStoppedNodes#testRetryCacheWithDownedOM`. Possibly due to https://issues.apache.org/jira/browse/RATIS-2605, the leadership's follower matchIndex advances slower now. However, the transferLeadership requires the target leader to be up-to-dated before start the transfer, otherwise the thread waits until timeout. However, even if we increase the timeout to 100 seconds, timeout still happens. Probably it means that there was not `AppendEntries` calls to increase the matchIndex. This PR proposes to force a `AppendEntries` by creating a key. This is a pure testing fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
