[
https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283419#comment-16283419
]
Tao Yang edited comment on YARN-7494 at 12/8/17 11:43 AM:
----------------------------------------------------------
Thanks for the patch. [~sunilg]
Some thoughts from my side:
* Agree with 2) from [~leftnoteasy]. Sorting requirement may be different in
some scenarios. For example, opportunistic containers would prefer considering
node utilization to unallocated resource. I think we should support expandable
sorting library and AppPlacementAllocator can choose from it.
* This patch iterates all partition nodes to create new
PartitionBasedCandidateNodeSet for every schedule process in
CapacityScheduler#getCandidateNodeSet. I think we can keep a single instance to
avoid always creating the same set.
* This patch remain as it is to iterates all nodes and trigger the schedule
process for every node in CapacityScheduler#schedule. Is it better to move
multiNodePlacementEnabled condition branch from
CapacityScheduler#getCandidateNodeSet to CapacityScheduler#schedule and
iterates all partitions to trigger the schedule process when
multi-node-placement enabled ?
was (Author: tao yang):
Thanks for the patch. [~sunilg]
Some thoughts from my side:
* Agree with 2) from [~leftnoteasy]. Sorting requirement may be different in
some scenarios. For example, opportunistic containers would prefer considering
node utilization to unallocated resource. I think we should support expandable
sorting library and AppPlacementAllocator can choose from it.
* This patch iterates all partition nodes to create new
PartitionBasedCandidateNodeSet for every schedule process in
CapacityScheduler#getCandidateNodeSet. I think we can keep a single instance to
avoid always creating the same set.
* This patch remain as it is to iterates all nodes and trigger the schedule
process for every node in CapacityScheduler#schedule. IIUC Is it better to move
multiNodePlacementEnabled condition branch from
CapacityScheduler#getCandidateNodeSet to CapacityScheduler#schedule and
iterates all partitions to trigger the schedule process when
multi-node-placement enabled ?
> Add muti node lookup support for better placement
> -------------------------------------------------
>
> Key: YARN-7494
> URL: https://issues.apache.org/jira/browse/YARN-7494
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Reporter: Sunil G
> Assignee: Sunil G
> Attachments: YARN-7494.v0.patch
>
>
> Instead of single node, for effectiveness we can consider a multi node lookup
> based on partition to start with.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
