[
https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16280902#comment-16280902
]
Wangda Tan commented on YARN-7494:
----------------------------------
Thanks [~sunilg] for working on this.
Looked at the implementation, some thoughts/suggestions:
1) This patch added a flag to ASC, I think we have many other requirements like
this, for example, applications may want to choose its own delay scheduling
parameters, etc. Instead of adding this to protocol, how about reading such
fields in AM launch context's environment (which is same as how we handle
docker container image, etc.). Part of the reason is, we don't need to change
applications to consume this feature. Most of the apps should be able to
specify customized envars for AM. (Any other ideas here?)
2) This patch added a getPreferredNodeIterator method to CandidateNodeSet,
however, I think we should keep CandidateNodeSet returns all available nodes
(like all nodes under the partition or whole cluster). Different
AppPlacementAllocator should implement its own sorting mechanism to allocate on
nodes.
3) To help other folks reviewing the implementation, I suggest adding a sample
multi-node placement allocator. Such as trying to make the allocation as packed
as possible (which essentially trying to allocate on nodes with less
utilization).
+ [~asuresh] / [~kkaranasos].
> Add muti node lookup support for better placement
> -------------------------------------------------
>
> Key: YARN-7494
> URL: https://issues.apache.org/jira/browse/YARN-7494
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Reporter: Sunil G
> Assignee: Sunil G
> Attachments: YARN-7494.v0.patch
>
>
> Instead of single node, for effectiveness we can consider a multi node lookup
> based on partition to start with.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
