[
https://issues.apache.org/jira/browse/HDFS-12648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16215895#comment-16215895
]
Hanisha Koneru commented on HDFS-12648:
---------------------------------------
Hi [~daryn], thanks for filing this improvement.
I would like to learn more about the problem. Specifically the following:
1. DN processes the commands from Namenode inline in the heartbeat response
thread. The next heartbeat would not be triggered before all the commands from
the previous heartbeat response are processed. So there will not be a backlog
of commands during next heartbeat.
2. Different commands will have different expected latencies. So the rate of
command processing might not be a clear indication of Datanode slowness.
3. Namenode would need to compare each type of command's processing time among
different Datanodes to detect outliers.
Please let me know if I have missed something here.
> DN should provide feedback to NN for throttling commands
> --------------------------------------------------------
>
> Key: HDFS-12648
> URL: https://issues.apache.org/jira/browse/HDFS-12648
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Affects Versions: 2.8.0
> Reporter: Daryn Sharp
> Assignee: Hanisha Koneru
>
> The NN should avoid sending commands to a DN with a high number of
> outstanding commands. The heartbeat could provide this feedback via perhaps
> a simple count of the commands or rate of processing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
