[
https://issues.apache.org/jira/browse/YARN-6599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16317379#comment-16317379
]
Arun Suresh commented on YARN-6599:
-----------------------------------
[~leftnoteasy], did a slightly deeper dive.
This is my understanding of the general flow - I'd break this down the 2 phases.
# When the app makes the allocate call: This is where you do the
updatePendingAsk etc for each SchedulingRequest. This is also where you
instantiate the AppPlacementAllocator and map it to the request. Looks like the
really interesting method is {{validateAndSetSchedulingRequest}}, which is
where you validate the placement constraints. This method sets the valid
targetAllocationTags etc.
# When the node heartbeats: At this point, in the leafQueue, we pick the
SchedulingRequest with highest priority and finally, we call the
{{canAllocate}} method which checks if the Node can be assigned to the
scheduling request based on the placementConstratint. right ?
Given the above, we should agree that:
This approach - while it ensures that allocation of a SchedulingRequest to a
node will guarantee that Constraints are NOT violated, It does nothing in the
way of trying to FIND the nodes that will meet the constraints.. agreed ?
My opinion - and something we should call out / document is that:
* If an app is more concerned about priority ordering of its requests, then we
can recommend using this approach.
* If the app the more concerned about an optimal placement, then the processor
route will give better results - since it activly tries to find nodes that
satisfy constraints by considering multiple schedulingRequests and multiple
nodes.
Thuoghts ?
Also some other comments:
* In the Scheduler, you are adding a new List<SchedulingRequest> parameter to
the allocate() method. Do you think, a better approach might be to create a
common superclass / interface which both the SchedulingRequest and
ResourceRequest extends and change the existing parameter to {{List<? extends
BaseRequest>}} ?
* If we do the above, then you wont have to duplicate methods like :
application.updateResourceRequests and normalizeSchedulingRequests
> Support rich placement constraints in scheduler
> -----------------------------------------------
>
> Key: YARN-6599
> URL: https://issues.apache.org/jira/browse/YARN-6599
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Attachments: YARN-6599-YARN-6592.003.patch,
> YARN-6599-YARN-6592.004.patch, YARN-6599-YARN-6592.005.patch,
> YARN-6599-YARN-6592.006.patch, YARN-6599-YARN-6592.007.patch,
> YARN-6599-YARN-6592.008.patch, YARN-6599-YARN-6592.wip.002.patch,
> YARN-6599.poc.001.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
