[
https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575020#comment-16575020
]
Weiwei Yang commented on YARN-7494:
-----------------------------------
Hi [~sunilg]
I think to get this not a CS only feature was from this comment commented by
[~leftnoteasy],
{quote}Ideally, MultiNodeLookupPolicy initialization should not determined by
Scheduler, and implementation of MultiNodeSortingManager should not bind to
specific scheduler.
{quote}
my opinion is if this causes burden ending up with changing a lot of files, it
seems better to limit this feature in CS. Just let the sorting manager be a
common service would be enough.
*LocalityAppPlacementAllocator*
line 90-94: we should not print this debug message if nodeLookupPolicy is null.
line 107: if multi-nodes lookup enabled and an app is giving an incorrect name,
then singleNode is null and nodeLookupPolicy is also null, then looks like
this will cause NPE.
*TestCapacityScheduler*
line 33: this seems to be an unintentional change
line 145-146: unused imports
*ResourceUsageMultiNodeLookupPolicy*
Currently this policy is using synchronized methods for thread safe, but this
could be sub-optimal. Scheduler alloc happens very fast, it may cause alloc
thread be waiting for sorting to be completed. This will get worse when there
is multiple alloc threads and a large number of nodes. In another word, we need
to decouple the SORT and GET.
*ResourceUsageMultiNodeLookupPolicyWithNoAutoSort*
I don't think we need another policy to achieve NoAutoSort. It is nature to
configure the {{sorting interval}} to 0 for
{{ResourceUsageMultiNodeLookupPolicy}} for this. How hard to get it done in
this way?
Hope it helps.
Thanks
> Add muti node lookup support for better placement
> -------------------------------------------------
>
> Key: YARN-7494
> URL: https://issues.apache.org/jira/browse/YARN-7494
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Reporter: Sunil Govindan
> Assignee: Sunil Govindan
> Priority: Major
> Attachments: YARN-7494.001.patch, YARN-7494.002.patch,
> YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch,
> YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch,
> YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch,
> YARN-7494.12.patch, YARN-7494.v0.patch, YARN-7494.v1.patch,
> multi-node-designProposal.png
>
>
> Instead of single node, for effectiveness we can consider a multi node lookup
> based on partition to start with.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
