[
https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283419#comment-16283419
]
Tao Yang edited comment on YARN-7494 at 12/8/17 11:47 AM:
----------------------------------------------------------
Thanks for the patch. [~sunilg]
Some thoughts from my side:
* Agree with 2) from [~leftnoteasy]. Sorting requirements may be different in
some scenarios. For example, opportunistic containers would prefer considering
node utilization to unallocated resource. I think we should support expandable
sorting library and AppPlacementAllocator can choose from it.
* This patch iterates all partition nodes to create new
{{PartitionBasedCandidateNodeSet}} instance for every schedule process in
{{CapacityScheduler#getCandidateNodeSet}}. I think we can keep a single
instance to avoid always creating it.
* This patch remains as it is to iterate all nodes and trigger the schedule
process for every node in {{CapacityScheduler#schedule}}. Is it better to move
{{multiNodePlacementEnabled}} check branch from
{{CapacityScheduler#getCandidateNodeSet}} to {{CapacityScheduler#schedule}} and
iterates all partitions to trigger the schedule process when
multi-node-placement enabled ?
was (Author: tao yang):
Thanks for the patch. [~sunilg]
Some thoughts from my side:
* Agree with 2) from [~leftnoteasy]. Sorting requirement may be different in
some scenarios. For example, opportunistic containers would prefer considering
node utilization to unallocated resource. I think we should support expandable
sorting library and AppPlacementAllocator can choose from it.
* This patch iterates all partition nodes to create new
{{PartitionBasedCandidateNodeSet}} instance for every schedule process in
{{CapacityScheduler#getCandidateNodeSet}}. I think we can keep a single
instance to avoid always creating the same set.
* This patch remains as it is to iterates all nodes and trigger the schedule
process for every node in CapacityScheduler#schedule. Is it better to move
multiNodePlacementEnabled condition branch from
{{CapacityScheduler#getCandidateNodeSet}} to {{CapacityScheduler#schedule}} and
iterates all partitions to trigger the schedule process when
multi-node-placement enabled ?
> Add muti node lookup support for better placement
> -------------------------------------------------
>
> Key: YARN-7494
> URL: https://issues.apache.org/jira/browse/YARN-7494
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Reporter: Sunil G
> Assignee: Sunil G
> Attachments: YARN-7494.v0.patch
>
>
> Instead of single node, for effectiveness we can consider a multi node lookup
> based on partition to start with.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
