[
https://issues.apache.org/jira/browse/YARN-412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roger Hoover updated YARN-412:
------------------------------
Attachment: YARN-412.patch
Please review this patch for the fix plus a unit test case
> FifoScheduler incorrectly checking for node locality
> ----------------------------------------------------
>
> Key: YARN-412
> URL: https://issues.apache.org/jira/browse/YARN-412
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Reporter: Roger Hoover
> Priority: Minor
> Labels: patch
> Attachments: YARN-412.patch
>
>
> In the FifoScheduler, the assignNodeLocalContainers method is checking if the
> data is local to a node by searching for the nodeAddress of the node in the
> set of outstanding requests for the app. This seems to be incorrect as it
> should be checking hostname instead. The offending line of code is 455:
> application.getResourceRequest(priority, node.getRMNode().getNodeAddress());
> Requests are formated by hostname (e.g. host1.foo.com) where as node
> addresses are a concatenation of hostname and command port (e.g.
> host1.foo.com:1234)
> In the CapacityScheduler, it's done using hostname. See
> LeafQueue.assignNodeLocalContainers, line 1129
> application.getResourceRequest(priority, node.getHostName());
> Note that this but does not affect the actual scheduling decisions by the
> FifoScheduler because even though it incorrect determines that a request is
> not local to the node, it will still schedule the request immediately because
> it's rack-local. However, this bug may be adversely affecting the reporting
> of job status by underreporting the number of tasks that were node local.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
