[
https://issues.apache.org/jira/browse/HDFS-8574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581389#comment-14581389
]
Walter Su commented on HDFS-8574:
---------------------------------
I don't know if it's necessary. If threshold is exceeded, the number of RPCs
equals to the number of storages. The number of storages is not quite many,
don't you think? What if threshold is exceeded per storage? The approach
increases RPCs actually.
> When block count exceeds dfs.blockreport.split.threshold, the block report
> are sent in one per message
> ------------------------------------------------------------------------------------------------------
>
> Key: HDFS-8574
> URL: https://issues.apache.org/jira/browse/HDFS-8574
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.7.0
> Reporter: Ajith S
> Assignee: Ajith S
>
> This piece of code in
> {{org.apache.hadoop.hdfs.server.datanode.BPServiceActor.blockReport()}}
> {code}
> // Send one block report per message.
> for (int r = 0; r < reports.length; r++) {
> StorageBlockReport singleReport[] = { reports[r] };
> DatanodeCommand cmd = bpNamenode.blockReport(
> bpRegistration, bpos.getBlockPoolId(), singleReport,
> new BlockReportContext(reports.length, r, reportId));
> numReportsSent++;
> numRPCs++;
> if (cmd != null) {
> cmds.add(cmd);
> }
> {code}
> is creating many cmds in case the block count exceeds the
> {{dfs.blockreport.split.threshold}} limit. A better way for this will be
> spliting the block reports in equal number of buckets of size
> {{dfs.blockreport.split.threshold}} therefore reducing the number of RPCs in
> block reporting
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
