[
https://issues.apache.org/jira/browse/HDDS-4577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18103610#comment-18103610
]
Siyao Meng commented on HDDS-4577:
----------------------------------
This method is failing again intermittently on master (2026-08), with a
different symptom than the original report:
{code}
[ERROR]
org.apache.hadoop.ozone.client.rpc.TestOzoneClientRetriesOnExceptions.testMaxRetriesByOzoneClient
-- Time elapsed: 16.54 s <<< ERROR!
java.io.IOException: Failed to execute command cmdType: CreateContainer
containerID: 1 ... createContainer { state: OPEN }
at
org.apache.hadoop.ozone.client.rpc.TestOzoneClientRetriesOnExceptions.testMaxRetriesByOzoneClient(TestOzoneClientRetriesOnExceptions.java:206)
Caused by: ... StateMachineException: ... Container creation failed because key
value container already exists
{code}
Likely a regression from HDDS-11020: it replaced the guard
{{Assumptions.assumeFalse(containerList.contains(containerID))}} with
{{assertThat(containerList.contains(containerID))}}. The AssertJ
{{assertThat(boolean)}} has no terminal (e.g. {{.isTrue()}}), so it asserts
nothing, and the loop now sends a second {{CreateContainer}} for a container
that SCM reused across block entries (more likely with the
{{RoundRobinPipelineChoosePolicy}} this test configures), which the datanode
rejects with CONTAINER_ALREADY_EXISTS. Restoring the {{assumeFalse(...)}} guard
fixes it. Not a product idempotency bug.
Seen in:
-
https://github.com/adoroszlai/ozone-build-results/blob/master/2026/08/10/52104/integration-client/hadoop-ozone/integration-test/org.apache.hadoop.ozone.client.rpc.TestOzoneClientRetriesOnExceptions.txt
-
https://github.com/adoroszlai/ozone-build-results/blob/master/2026/08/01/51813/integration-client/hadoop-ozone/integration-test/org.apache.hadoop.ozone.client.rpc.TestOzoneClientRetriesOnExceptions.txt
-
https://github.com/adoroszlai/ozone-build-results/blob/master/2026/08/03/51849/integration-client/hadoop-ozone/integration-test/org.apache.hadoop.ozone.client.rpc.TestOzoneClientRetriesOnExceptions.txt
-
https://github.com/adoroszlai/ozone-build-results/blob/master/2026/08/05/51982/integration-client/hadoop-ozone/integration-test/org.apache.hadoop.ozone.client.rpc.TestOzoneClientRetriesOnExceptions.txt
> Intermittent failure in
> TestOzoneClientRetriesOnException#testMaxRetriesByOzoneClient
> -------------------------------------------------------------------------------------
>
> Key: HDDS-4577
> URL: https://issues.apache.org/jira/browse/HDDS-4577
> Project: Apache Ozone
> Issue Type: Bug
> Components: test
> Reporter: Attila Doroszlai
> Assignee: Attila Doroszlai
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> TestOzoneClientRetriesOnException#testMaxRetriesByOzoneClient intermittently
> fails with:
> {code:title=https://github.com/elek/ozone-build-results/blob/master/2020/12/08/4407/it-client/hadoop-ozone/integration-test/org.apache.hadoop.ozone.client.rpc.TestOzoneClientRetriesOnException.txt}
> testMaxRetriesByOzoneClient(org.apache.hadoop.ozone.client.rpc.TestOzoneClientRetriesOnException)
> Time elapsed: 42.789 s <<< FAILURE!
> java.lang.AssertionError: Expected exception not thrown
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.hadoop.ozone.client.rpc.TestOzoneClientRetriesOnException.testMaxRetriesByOzoneClient(TestOzoneClientRetriesOnException.java:235)
> {code}
> The problem happens if the number of distinct containers used for the key is
> less than 4. In this case max retries (of 3) is never reached.
> Normal execution:
> {code}
> block: conID: 1 locID: 105357240184995843 bcsId: 0
> block: conID: 2 locID: 105357240185061380 bcsId: 0
> block: conID: 3 locID: 105357240185061381 bcsId: 0
> block: conID: 4 locID: 105357240185061382 bcsId: 0
> {code}
> Failing execution:
> {code}
> block: conID: 1 locID: 105357233754603523 bcsId: 0
> block: conID: 2 locID: 105357233754603524 bcsId: 0
> block: conID: 1 locID: 105357233754669061 bcsId: 0
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
