[
https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16545031#comment-16545031
]
Sunil Govindan commented on YARN-7494:
--------------------------------------
Thanks [~cheersyang]. Addressed the comments. Pls find some more details below.
bq. Is "yarn.scheduler.capacity.multi-node-sorting.policy" necessary? We can
use "yarn.scheduler.capacity.root.multi-node-sorting.policy" to set global
sorting policy for CS right?
This is inline with existing config model in CS. All such cs level configs are
starting with ** *yarn.scheduler.capacity.* for eg:
maximum-am-resource-percent etc
I think this is a better model in align with existing configs.
bq.line 2197 - 2201: the check is not necessary, policyClassName cannot be null
as there is default value given.
We expect user to configure the MultiNode policy name per queue (or we will
take from cluster level). multi-node-sorting.policy
{color:#333333}config is used for this. Once the policy name is given, we
expect user to provide the class name for that policy and hence the validation
is added. I did some more hardening and refactoring to make it more cleaner.
However we need that if check as custom policy name doesnt have any
defaults.{color}
bq.line 2203: looks like the sorting interval is a global setting, is it better
to set per-policy? E.g
"yarn.scheduler.capacity.multi-node-sorting.policy.<policyName>.[sorting-task.interval.ms|http://sorting-task.interval.ms/]";.
This could be a followup task if you agree with this suggestion.
Ys. lets track this separately.
> Add muti node lookup support for better placement
> -------------------------------------------------
>
> Key: YARN-7494
> URL: https://issues.apache.org/jira/browse/YARN-7494
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Reporter: Sunil Govindan
> Assignee: Sunil Govindan
> Priority: Major
> Attachments: YARN-7494.001.patch, YARN-7494.002.patch,
> YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch,
> YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch,
> YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.v0.patch,
> YARN-7494.v1.patch, multi-node-designProposal.png
>
>
> Instead of single node, for effectiveness we can consider a multi node lookup
> based on partition to start with.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
