[
https://issues.apache.org/jira/browse/HDFS-7923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14563456#comment-14563456
]
Colin Patrick McCabe commented on HDFS-7923:
--------------------------------------------
I posted a new patch with some changes from the previous approach. Since
Datanodes can go away at any time after the NN gives them the green light, this
patch adds the concept of leases for block reports. Leases have a fixed time
length... if the DN can't send its block report within that time, it loses the
lease. I also added a new fault injection framework to monitor what is going
on in the BlockManager. There was some milliseconds / seconds confusion in the
existing initial block report delay code that I fixed (might want to split this
off into a separate JIRA...)
> The DataNodes should rate-limit their full block reports by asking the NN on
> heartbeat messages
> -----------------------------------------------------------------------------------------------
>
> Key: HDFS-7923
> URL: https://issues.apache.org/jira/browse/HDFS-7923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-7923.000.patch, HDFS-7923.001.patch,
> HDFS-7923.002.patch, HDFS-7923.003.patch
>
>
> The DataNodes should rate-limit their full block reports. They can do this
> by first sending a heartbeat message to the NN with an optional boolean set
> which requests permission to send a full block report. If the NN responds
> with another optional boolean set, the DN will send an FBR... if not, it will
> wait until later. This can be done compatibly with optional fields.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
