[
https://issues.apache.org/jira/browse/HDFS-14651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16977657#comment-16977657
]
Lisheng Sun commented on HDFS-14651:
------------------------------------
Thanks [~linyiqun] for deep review.
{quote}When we remove one dead node, can we also remove the dead node from
dfsInputStreamNodes and DFSInputStream local dead nodes?
{quote}
DeadNodeDetector#removeNodeFromDeadNode can remove the dead node from
dfsInputStreamNodes and deanodes as follow. Its name may not be very good, but
I have not thought of it better.
{code:java}
/**
* Remove dead node from dfsInputStreamNodes#dfsInputStream and deadNodes.
*/
public synchronized void removeNodeFromDeadNode(DatanodeInfo datanodeInfo) {
for (Map.Entry<DFSInputStream, HashSet<DatanodeInfo>> entry :
dfsInputStreamNodes.entrySet()) {
Set<DatanodeInfo> datanodeInfos = entry.getValue();
datanodeInfos.remove(datanodeInfo);
}
removeFromDead(datanodeInfo);
}
{code}
i think that remove from dead node from DFSInputStream local dead nodes should
not DeadNodeDetector's work. DeadNodeDetector's purpose is to handler sharing
deadnode in same client.
{quote}We should have corresponding close operations of detector and stop these
threads.
{quote}
we have set these threads damon, so we need not stop these threads.
{quote}I noticed that here we only have one type CHECK_DEAD not other probe
type like suspicious node type? Do we still need to define this type? Seems
it's okay that we don't need to add suspicious state.
{quote}
HDFS-14649 will add CHECK_SUSPICIOUS type. it is executed in different probe
thread pool by different probe type. so we need define this type.
{quote}dfs.client.deadnode.detection.dead.threads -->
dfs.client.deadnode.detection.probe.threads, description: The maximum number of
threads to use for probing dead node.
dfs.client.deadnode.detection.dead.interval -->
dfs.client.deadnode.detection.probe.interval.ms. description: Interval time in
milliseconds for probing dead node behavior.
{quote}
since we will add probe suspicious node, i define these variable name is
dfs.client.deadnode.detection.dead.threads
-->dfs.client.deadnode.detection.probe.deadnode.threads
dfs.client.deadnode.detection.dead.interval -->
dfs.client.deadnode.detection.probe.deadnode.interval.ms
And uploaded the v004 patch.
Please correct me if i was wrong. Thank you a lot [~linyiqun]
> DeadNodeDetector checks dead node periodically
> ----------------------------------------------
>
> Key: HDFS-14651
> URL: https://issues.apache.org/jira/browse/HDFS-14651
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Lisheng Sun
> Assignee: Lisheng Sun
> Priority: Major
> Attachments: HDFS-14651.001.patch, HDFS-14651.002.patch,
> HDFS-14651.003.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
