Chi-Hsuan Huang created HDDS-17405:
--------------------------------------
Summary: Intermittent failure in
TestOzoneManagerHAFollowerReadWithStoppedNodes#testFollowerReadOmProxyProviderFailoverOnConnectionFailure
Key: HDDS-17405
URL: https://issues.apache.org/jira/browse/HDDS-17405
Project: Apache Ozone
Issue Type: Sub-task
Components: OM HA, test
Reporter: Chi-Hsuan Huang
{code}
org.apache.hadoop.ozone.om.TestOzoneManagerHAFollowerReadWithStoppedNodes.testFollowerReadOmProxyProviderFailoverOnConnectionFailure
-- Time elapsed: 15.49 s <<< ERROR!
org.apache.hadoop.ipc_.RemoteException(org.apache.ratis.protocol.exceptions.ReadIndexException):
omNode-1@group-523986131536: Leader is unknown.
at
org.apache.ratis.server.impl.RaftServerImpl.sendReadIndexAsync(RaftServerImpl.java:1110)
...
at
org.apache.hadoop.ozone.om.TestOzoneManagerHAFollowerReadWithStoppedNodes.testFollowerReadOmProxyProviderFailoverOnConnectionFailure(TestOzoneManagerHAFollowerReadWithStoppedNodes.java:252)
{code}
h3. Root cause
Same as HDDS-16843, in the follower read proxy test. The test stops the current
proxy OM, which was the leader in both failing runs, and immediately sends a
{{listVolumes}}. The first election after that ended in a split vote, so the
client used up its 5 failovers, getting "Leader is unknown" from both remaining
OMs, before a new leader was elected 0.2-0.3 s later.
h3. Fix
Call {{waitForLeaderToBeReady()}} after stopping the OM, as done in HDDS-16843
for {{testLeaderOmProxyProviderFailoverOnConnectionFailure}}. The client still
points at the stopped OM, so the next request still exercises the failover.
- https://github.com/apache/ozone/actions/runs/37302015763/job/111741681965
- https://github.com/apache/ozone/actions/runs/38766809137/job/116603516139
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
