[
https://issues.apache.org/jira/browse/HADOOP-5034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12665975#action_12665975
]
dhruba borthakur commented on HADOOP-5034:
------------------------------------------
+1 for putting this into 0.19. Very helpful patch.
> NameNode should send both both replication and deletion requests to DataNode
> in one reply to a heartbeat
> --------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5034
> URL: https://issues.apache.org/jira/browse/HADOOP-5034
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.21.0
>
> Attachments: blockTransferInvalidate.patch
>
>
> Currently NameNode favors block replication requests over deletion requests.
> On reply to a heartbeat, NameNode does not send a block deletion request
> unless there is no block replication request.
> This brings a problem when a near-full cluster loses a bunch of DataNodes. In
> react to the DataNode loss, NameNode starts to replicate blocks. However,
> replication takes a lot of cpu and a lot of replications fail because of the
> lack of disk space. So the administrator tries to delete some DFS files to
> free up space. However, block deletion requests get delayed for very long
> time because it takes a long time to drain the block replication requests for
> most DataNodes.
> I'd like to propose to let NameNode to send both replication requests and
> deletion requests to DataNodes in one reply to a heartbeat. This also implies
> that the replication monitor should schedule both replication and deletion
> work in one iteration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
