[
https://issues.apache.org/jira/browse/HDFS-13571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884389#comment-16884389
]
Yiqun Lin commented on HDFS-13571:
----------------------------------
Hi [~leosun08], the idea makes sense to me. I did a quick through for your
patch, it seems a little large. If we want to backport to the trunk, we need to
separate this big patch to sub-tasks. And it will be convenient for others to
review. But before creating sub-tasks for this feature, can you make a simple
summary for this implementation? For example, introducing the the mechanism of
the core implementation DeadNodeDetector and how it works with DFSClient and
DFSInputStream. This is the key information we want to get.
> Dead DataNode Detector
> ----------------------
>
> Key: HDFS-13571
> URL: https://issues.apache.org/jira/browse/HDFS-13571
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 2.4.0, 2.6.0, 3.0.2
> Reporter: Gang Xie
> Assignee: Lisheng Sun
> Priority: Minor
> Attachments: HDFS-13571-2.6.diff
>
>
> Currently, the information of the dead datanode in DFSInputStream in stored
> locally. So, it could not be shared among the inputstreams of the same
> DFSClient. In our production env, every days, some datanodes dies with
> different causes. At this time, after the first inputstream blocked and
> detect this, it could share this information to others in the same DFSClient,
> thus, the ohter inputstreams are still blocked by the dead node for some
> time, which could cause bad service latency.
> To eliminate this impact from dead datanode, we designed a dead datanode
> detector, which detect the dead ones in advance, and share this information
> among all the inputstreams in the same client. This improvement has being
> online for some months and works fine. So, we decide to port to the 3.0 (the
> version used in our production env is 2.4 and 2.6).
> I will do the porting work and upload the code later.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
