[
https://issues.apache.org/jira/browse/HDFS-7923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17647866#comment-17647866
]
Yanlei Yu commented on HDFS-7923:
---------------------------------
hi，[~cmccabe] ，in my 1000+ cluster test, I found that during namenode restart,
if datanode does not connect to namenode because namenode is busy, it will
print IOException in offerService, However, forceFullBr is already false in the
next loop, which causes the datanode to fail to send FBR during the namenode
restart. As a result, the namenode restarts slowly. The block reporting phase
takes about 20 hours. I can set forceFullBr to true after the exception is
received so that the datanode can send the FBR in a timely manner. How do you
feel?
> The DataNodes should rate-limit their full block reports by asking the NN on
> heartbeat messages
> -----------------------------------------------------------------------------------------------
>
> Key: HDFS-7923
> URL: https://issues.apache.org/jira/browse/HDFS-7923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 2.8.0
> Reporter: Colin McCabe
> Assignee: Colin McCabe
> Priority: Major
> Fix For: 2.8.0, 3.0.0-alpha1
>
> Attachments: HDFS-7923.000.patch, HDFS-7923.001.patch,
> HDFS-7923.002.patch, HDFS-7923.003.patch, HDFS-7923.004.patch,
> HDFS-7923.006.patch, HDFS-7923.007.patch
>
>
> The DataNodes should rate-limit their full block reports. They can do this
> by first sending a heartbeat message to the NN with an optional boolean set
> which requests permission to send a full block report. If the NN responds
> with another optional boolean set, the DN will send an FBR... if not, it will
> wait until later. This can be done compatibly with optional fields.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
