[
https://issues.apache.org/jira/browse/HDFS-12648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16279323#comment-16279323
]
Daryn Sharp commented on HDFS-12648:
------------------------------------
I think we need this support before HDFS-12647. It doesn't make much sense for
that jira to not send heartbeats when the queue is full.
I think the simplest answer is the DN heartbeats the max number of commands it
will accept. The NN simply hands out no more than that. Replication commands
take priority over invalidation work. The former implies data is at risk, the
former is just trying to clean up space. Work is already computed in that
order anyway.
The max commands can default to Integer.MAX_VALUE for backwards compatibility
with old nodes since that's effectively what it is today. Then when HDFS-12647
makes the DN smarter, we'll have the NN support already in place.
> DN should provide feedback to NN for throttling commands
> --------------------------------------------------------
>
> Key: HDFS-12648
> URL: https://issues.apache.org/jira/browse/HDFS-12648
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Affects Versions: 2.8.0
> Reporter: Daryn Sharp
> Assignee: Hanisha Koneru
>
> The NN should avoid sending commands to a DN with a high number of
> outstanding commands. The heartbeat could provide this feedback via perhaps
> a simple count of the commands or rate of processing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
