[
https://issues.apache.org/jira/browse/HDFS-11313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816933#comment-15816933
]
Vinitha Reddy Gankidi commented on HDFS-11313:
----------------------------------------------
[~shv] This idea seems promising. I would like to work on it. I wanted to note
that HDFS-7923 is related in the sense that the blocks reports by the DN are
sent only when the NN gives the signal. Even with this patch, the issue of
processing large DN reports under a global namespace lock still remains.
> Segmented Block Reports
> -----------------------
>
> Key: HDFS-11313
> URL: https://issues.apache.org/jira/browse/HDFS-11313
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode, namenode
> Affects Versions: 2.6.2
> Reporter: Konstantin Shvachko
>
> Block reports from a single DataNode can be currently split into multiple
> RPCs each reporting a single DataNode storage (disk). The reports are still
> large since disks are getting bigger. Splitting blockReport RPCs into
> multiple smaller calls would improve NameNode performance and overall HDFS
> stability.
> This was discussed in multiple jiras. Here the approach is to let NameNode
> divide blockID space into segments and then ask DataNodes to report replicas
> in a particular range of IDs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
