[
https://issues.apache.org/jira/browse/HDFS-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17433671#comment-17433671
]
Janus Chow commented on HDFS-16275:
-----------------------------------
[~ayushtkn] Thanks for the comment.
I think we do like to have data locality, only not a too hot one. IMHO when the
node is not too hot, the locality should boost the performance. The change from
default "false" is mainly for cooling the node down.
> [HDFS] Enable considerLoad for localWrite
> -----------------------------------------
>
> Key: HDFS-16275
> URL: https://issues.apache.org/jira/browse/HDFS-16275
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Janus Chow
> Assignee: Janus Chow
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently when client is on the same machine of a datanode, it will try to
> write to the local machine regardless of the load of the datanode, that is
> the xceiverCount.
> In our production cluster, datanode and Nodemanager are running on the same
> server, so when there are heavy jobs running on a labeled queue, the
> corresponding datanodes will have higher xceiverCounts than other datanodes.
> When other clients are trying to write, the exception of "could only be
> replicated to 0 nodes" would be thrown.
> This ticket is to enable considerLoad to avoid the hot local write.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
