Shuyan Zhang created HDFS-17204:
-----------------------------------
Summary: EC: Reduce unnecessary log when processing excess
redundancy.
Key: HDFS-17204
URL: https://issues.apache.org/jira/browse/HDFS-17204
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Shuyan Zhang
This is a follow-up of
[HDFS-16964|https://issues.apache.org/jira/browse/HDFS-16964]. We now avoid
stale replicas when dealing with redundancy. This may result in redundant
replicas not being in the `nonExcess` set when we enter
`BlockManager#chooseExcessRedundancyStriped` (because the datanode where the
redundant replicas are located has not send FBR yet, so those replicas are
filtered out and not added to the `nonExcess` set). A further result is that no
excess storage type is selected and the log "excess types chosen for block..."
is printed. When a failover occurs, a large number of datanodes become stale,
which causes NameNodes to print a large number of unnecessary logs.
This issue needs to be fixed, otherwise the performance after failover will be
affected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
