Doris Gu created HDFS-11104:
-------------------------------
Summary: BlockPlacementPolicyDefault choose favoredNodes in turn
which may cause imbalance
Key: HDFS-11104
URL: https://issues.apache.org/jira/browse/HDFS-11104
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Doris Gu
if client transfer favoredNodes when it writes files into hdfs,chooseTarget in
BlockPlacementPolicyDefault prior chooseTarget in turn:
{quote}
DatanodeStorageInfo[] chooseTarget(String src,
int numOfReplicas,
Node writer,
Set<Node> excludedNodes,
long blocksize,
List<DatanodeDescriptor> favoredNodes,
BlockStoragePolicy storagePolicy) {
try {
...
*for (int i = 0; i < favoredNodes.size() && results.size() < numOfReplicas;
i++)* {
DatanodeDescriptor favoredNode = favoredNodes.get(i);
// Choose a single node which is local to favoredNode.
// 'results' is updated within chooseLocalNode
final DatanodeStorageInfo target = chooseLocalStorage(favoredNode,
favoriteAndExcludedNodes, blocksize, maxNodesPerRack,
results, avoidStaleNodes, storageTypes, false);
...
{quote}
why not shuffle it?
{quote}
*for (DatanodeDescriptor favoredNode :
DFSUtil.shuffle(favoredNodes.toArray(new
DatanodeDescriptor[favoredNodes.size()]))) *
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
