[
https://issues.apache.org/jira/browse/YARN-5949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15987979#comment-15987979
]
Jonathan Hung commented on YARN-5949:
-------------------------------------
Thanks for the comments [~leftnoteasy], had a few questions/comments:
bq. we should avoid directly call CapacityScheduler in
removeQueue/addQueue/updateQueue as well
Seems reasonable, we can also use the getQueueInfo API here, this seems to be
the only way to replicate the "getQueue/setQueues" functionality from
CapacityScheduler/CapacitySchedulerConfiguration.
bq. QueueAdminConfigurationMutationPolicy use regex to parse the configs and
get existing queue names, we should avoid doing this.
This makes sense, should just be able to either grab the second-to-last
component of the key, or the component before
"accessible-node-labels"/"ordering-policy". I think we can also search for
"root", if it is not there then assume it is a global config change.
bq. Also we should get existing queues when necessary, I think we can add an
API to reinitialize ConfigurationMutationPolicy.
As long as we access these queues via YarnScheduler#getQueueInfo, is this API
still necessary? When the scheduler is reinitialized and the next mutation
comes in, it will check against the queues from the most recent
reinitialization.
bq. Please make sure QueueAdminConfigurationMutationPolicy can handle invalid
configs (like "x.y.z" and x is not a root queue)
This is not implemented yet, but I was thinking of handling this in
RMWebServices, there are some cases that have not been handled (e.g. updating
config for a queue which doesn't exist shouldn't be allowed, right now it
"succeeds" silently). So we can address these cases in a separate jira.
bq. Is it possible to change a global config (such as
yarn.scheduler.capacity.maximum-applications), if so, is following check enough?
Yes you're right, this is not handled yet, in fact there is still some handling
we need to do in RMWebServices for global configs, we can address this in a
separate jira as well. If global config is found then we should check for admin
permission as you mentioned.
> Add pluggable configuration policy interface as a component of
> MutableCSConfigurationProvider
> ---------------------------------------------------------------------------------------------
>
> Key: YARN-5949
> URL: https://issues.apache.org/jira/browse/YARN-5949
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Jonathan Hung
> Assignee: Jonathan Hung
> Attachments: YARN-5949-YARN-5734.001.patch
>
>
> This will allow different policies to customize how/if configuration changes
> should be applied (for example, a policy might restrict whether a
> configuration change by a certain user is allowed). This will be enforced by
> the MutableCSConfigurationProvider.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
