Siyao Meng created HDDS-17328:
---------------------------------
Summary: EC read waits about 2 minutes on an unresponsive datanode
before switching to reconstruction
Key: HDDS-17328
URL: https://issues.apache.org/jira/browse/HDDS-17328
Project: Apache Ozone
Issue Type: Improvement
Components: EC Client
Reporter: Siyao Meng
Assignee: Siyao Meng
{{ECBlockInputStream}} reads each data index through a single-datanode
pipeline. On a failed read, its {{BlockInputStream}} retries
{{ozone.client.read.max.retries}} (3) times with
{{ozone.client.read.retry.interval}} (1s), and a deadline exceeded error is not
treated as a connectivity failure, so it retries the same datanode. Only then
does {{BadDataLocationException}} make {{ECBlockInputStreamProxy}} switch to
reconstruction read. With an unresponsive datanode (not refusing connections)
this is 4 × 30s ({{ozone.client.read.timeout}}) + 3 × 1s, about 2 minutes, for
each block with an index on that datanode (by calculation). A datanode that
refuses connections fails in a few seconds.
Proposed fix: switch to reconstruction read on the first timeout from a
datanode, instead of retrying the same datanode.
Related: HDDS-16871 (share failed datanodes across the blocks of a key),
HDDS-16868
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
