[
https://issues.apache.org/jira/browse/TEZ-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002646#comment-14002646
]
Siddharth Seth commented on TEZ-1039:
-------------------------------------
The question really is - in which scenario would doAssignAll (invoked by
allocateTask), and assignDelayedContainer(after resetting locality match levels
- done by deallocateTask) not end up hitting the correct scheduling paths to
ensure that a node local match is adequate.
doAssignAll walks all current containers starting with node level matches.
assignDelayedContainer (with locality level reset) should ensure that specific
container gets matched if a request with affinity exists.
There may be some problems if there's a mix of affinity and non-affinity based
requests at the same priority level. Also possibly if the affinity-based
requests are waiting for other tasks to complete before they are scheduled (so
not scheduled immediately after the task they are being affinitized to). Even
in this case, I'd expect a de-allocate call to assign the newly available
container correctly.
> Add Container locality to TaskScheduler
> ---------------------------------------
>
> Key: TEZ-1039
> URL: https://issues.apache.org/jira/browse/TEZ-1039
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-1039.1.patch, TEZ-1039.2.patch
>
>
> Currently, the task scheduler support node and rack locality. Proposal is to
> add another level - container locality (given a container) which means first
> try to assign to the container, then fall back to the container's node, rack
> and so on.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
