[
https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16577772#comment-16577772
]
Weiwei Yang commented on YARN-7494:
-----------------------------------
Thanks [~sunilg] for the updates, please see my followup comments below:
ResourceUsageMultiNodeLookupPolicy.java:
{code:java}
Iterator<N> getPreferredNodeIterator(Collection<N> nodes, String partition,
boolean isAutoSortEnabled);
{code}
I understand you have added this to make background-sort completely controlled
by the sorting interval argument, but it complex the API. I think we can add a
wrapper in {{MultiNodeSortingManager#getPreferredNodeIterator}}, and let app
placement allocator call this instead. This way, we can remove {{boolean
isAutoSortEnabled}} argument because the manager is able to tell if the sorting
thread is running or not (or simply checking the sorting interval value). This
also gives the benefit that we don't need to expose the \{{sorter}} class to
app allocator.
Minor things to fix:
# ResourceUsageMultiNodeLookupPolicy.java: replace line 23 import star
# TestCapacityScheduler.java: replace line 33 import start
# TestCapacitySchedulerNodeLabelUpdate.java: remove unused imports at line 25
and 49
# ResourceUsageMultiNodeLookupPolicy.java: replace line 23 import start
# ApplicationSchedulingConfig.java: line 20, 24 unnecessary changes
# FiCaSchedulerApp.java: line 174-187: can we put this in a private method e.g
{{updateNodesSortingPolicyIfNecessary()}}?
Other looks good to me! Please let me know if this makes sense.
Thanks!
> Add muti node lookup support for better placement
> -------------------------------------------------
>
> Key: YARN-7494
> URL: https://issues.apache.org/jira/browse/YARN-7494
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Reporter: Sunil Govindan
> Assignee: Sunil Govindan
> Priority: Major
> Attachments: YARN-7494.001.patch, YARN-7494.002.patch,
> YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch,
> YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch,
> YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch,
> YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch,
> YARN-7494.15.patch, YARN-7494.v0.patch, YARN-7494.v1.patch,
> multi-node-designProposal.png
>
>
> Instead of single node, for effectiveness we can consider a multi node lookup
> based on partition to start with.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
