[
https://issues.apache.org/jira/browse/HADOOP-5034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12665983#action_12665983
]
Hairong Kuang commented on HADOOP-5034:
---------------------------------------
> I think it will simpler if we make heartbeat reply to contain list or array
> of commands. This way we don't need new command types.
I like this idea a lot. This makes the solution simpler and make the protocol
easy to extend later on.
> putting this into 0.19.
Not a problem for me. This patch has a protocol change. If the community wants
it in 0.19, maybe we can make an exception. :-)
> NameNode should send both both replication and deletion requests to DataNode
> in one reply to a heartbeat
> --------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5034
> URL: https://issues.apache.org/jira/browse/HADOOP-5034
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.21.0
>
> Attachments: blockTransferInvalidate.patch
>
>
> Currently NameNode favors block replication requests over deletion requests.
> On reply to a heartbeat, NameNode does not send a block deletion request
> unless there is no block replication request.
> This brings a problem when a near-full cluster loses a bunch of DataNodes. In
> react to the DataNode loss, NameNode starts to replicate blocks. However,
> replication takes a lot of cpu and a lot of replications fail because of the
> lack of disk space. So the administrator tries to delete some DFS files to
> free up space. However, block deletion requests get delayed for very long
> time because it takes a long time to drain the block replication requests for
> most DataNodes.
> I'd like to propose to let NameNode to send both replication requests and
> deletion requests to DataNodes in one reply to a heartbeat. This also implies
> that the replication monitor should schedule both replication and deletion
> work in one iteration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
