David Mollitor created MAPREDUCE-7243:
-----------------------------------------
Summary: Randomize List of Resources to Localize
Key: MAPREDUCE-7243
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7243
Project: Hadoop Map/Reduce
Issue Type: New Feature
Components: nodemanager, performance
Reporter: David Mollitor
Assignee: David Mollitor
https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/LocalResourceBuilder.java
Add a new parameter to {{LocalResourceBuilder}} that allows the list of
resources to be shuffled randomly. This will allow the Localizer to spread the
load of requests so that not all of the NodeManagers are requesting to localize
the same files, in the same order, from the same DataNodes,
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
