[
https://issues.apache.org/jira/browse/YARN-412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13630075#comment-13630075
]
Hudson commented on YARN-412:
-----------------------------
Integrated in Hadoop-Mapreduce-trunk #1397 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1397/])
YARN-412. Fixed FifoScheduler to check hostname of a NodeManager rather
than its host:port during scheduling which caused incorrect locality for
containers. Contributed by Roger Hoover. (Revision 1467244)
Result = SUCCESS
acmurthy :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1467244
Files :
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
*
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
*
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java
> FifoScheduler incorrectly checking for node locality
> ----------------------------------------------------
>
> Key: YARN-412
> URL: https://issues.apache.org/jira/browse/YARN-412
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Reporter: Roger Hoover
> Assignee: Roger Hoover
> Priority: Minor
> Labels: patch
> Fix For: 2.0.4-alpha
>
> Attachments: YARN-412.patch
>
>
> In the FifoScheduler, the assignNodeLocalContainers method is checking if the
> data is local to a node by searching for the nodeAddress of the node in the
> set of outstanding requests for the app. This seems to be incorrect as it
> should be checking hostname instead. The offending line of code is 455:
> application.getResourceRequest(priority, node.getRMNode().getNodeAddress());
> Requests are formated by hostname (e.g. host1.foo.com) whereas node addresses
> are a concatenation of hostname and command port (e.g. host1.foo.com:1234)
> In the CapacityScheduler, it's done using hostname. See
> LeafQueue.assignNodeLocalContainers, line 1129
> application.getResourceRequest(priority, node.getHostName());
> Note that this bug does not affect the actual scheduling decisions made by
> the FifoScheduler because even though it incorrect determines that a request
> is not local to the node, it will still schedule the request immediately
> because it's rack-local. However, this bug may be adversely affecting the
> reporting of job status by underreporting the number of tasks that were node
> local.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
