[
https://issues.apache.org/jira/browse/HDDS-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17839890#comment-17839890
]
Ethan Rose commented on HDDS-9754:
----------------------------------
I have an integration test on my fork based off master at 89afdd320c that still
repros this issue: https://github.com/errose28/ozone/tree/pipeline-retry-repro
The test does the following:
# Injects an error to fail datanode requests for container 1 only. Requests for
all other containers should still pass.
# Removes code on SCM to handle pipeline close requests from datanodes. This
keeps the faulty pipeline open.
** While this would usually eventually get resolved due to the fix in
HDDS-9735, the client should retry on a different pipeline in the meantime. By
holding on to this pipeline, we can show that the client is blocked on the
stalled pipeline and does not try a new one.
# Adds a new test in {{TestMiniOzoneCluster#testPipelineRetry}} that waits for
3 Ratis pipelines to become available and then writes data.
** If the above server side modifications are not present, the test passes.
** The test is expected to pass even with the above modifications because the
client would retry the failed write on a new container or pipeline.
** Instead, the test stalls forever. The client keeps retrying with the same
container and pipeline.
> Ozone Client Does not Retry on a New Pipeline
> ---------------------------------------------
>
> Key: HDDS-9754
> URL: https://issues.apache.org/jira/browse/HDDS-9754
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Datanode, SCM
> Reporter: Ethan Rose
> Priority: Major
> Attachments: pipeline_retry_repro.diff
>
>
> During testing, we observed that when the client was given bad pipeline, it
> continued trying to write to it forever, making it stuck indefinitely. The
> client should eventually give up on the write and get a different pipeline
> from OM.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
