[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283419#comment-16283419 ]
Tao Yang edited comment on YARN-7494 at 12/8/17 11:45 AM: ---------------------------------------------------------- Thanks for the patch. [~sunilg] Some thoughts from my side: * Agree with 2) from [~leftnoteasy]. Sorting requirement may be different in some scenarios. For example, opportunistic containers would prefer considering node utilization to unallocated resource. I think we should support expandable sorting library and AppPlacementAllocator can choose from it. * This patch iterates all partition nodes to create new {{PartitionBasedCandidateNodeSet}} instance for every schedule process in {{CapacityScheduler#getCandidateNodeSet}}. I think we can keep a single instance to avoid always creating the same set. * This patch remains as it is to iterates all nodes and trigger the schedule process for every node in CapacityScheduler#schedule. Is it better to move multiNodePlacementEnabled condition branch from {{CapacityScheduler#getCandidateNodeSet}} to {{CapacityScheduler#schedule}} and iterates all partitions to trigger the schedule process when multi-node-placement enabled ? was (Author: tao yang): Thanks for the patch. [~sunilg] Some thoughts from my side: * Agree with 2) from [~leftnoteasy]. Sorting requirement may be different in some scenarios. For example, opportunistic containers would prefer considering node utilization to unallocated resource. I think we should support expandable sorting library and AppPlacementAllocator can choose from it. * This patch iterates all partition nodes to create new PartitionBasedCandidateNodeSet for every schedule process in CapacityScheduler#getCandidateNodeSet. I think we can keep a single instance to avoid always creating the same set. * This patch remain as it is to iterates all nodes and trigger the schedule process for every node in CapacityScheduler#schedule. Is it better to move multiNodePlacementEnabled condition branch from CapacityScheduler#getCandidateNodeSet to CapacityScheduler#schedule and iterates all partitions to trigger the schedule process when multi-node-placement enabled ? > Add muti node lookup support for better placement > ------------------------------------------------- > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler > Reporter: Sunil G > Assignee: Sunil G > Attachments: YARN-7494.v0.patch > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org