[
https://issues.apache.org/jira/browse/YARN-5949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994089#comment-15994089
]
Wangda Tan commented on YARN-5949:
----------------------------------
Thanks [~jhung],
bq. This makes sense, should just be able to either grab the second-to-last
component of the key, or the component before
"accessible-node-labels"/"ordering-policy". I think we can also search for
"root", if it is not there then assume it is a global config change.
I think we can handle it in this way:
There're a set of known queue paths like \{"root.queueA", "root.queueA.A1\}.
For a given config key to change, first we need to remove the common prefix
("yarn.scheduler.capacity."), do the longest prefix match in the known queue
paths.
- If we can find any non-empty common prefix, check the queue's accessibilities.
- If we cannot find, this is a global config, check admin permission.
This approach doesn't need to handle special options like
"accessible-node-labels", and don't need to use "root" to index starting of
queue path, to me it is not a safe approach.
bq. As long as we access these queues via YarnScheduler#getQueueInfo, is this
API still necessary? When the scheduler is reinitialized and the next mutation
comes in, it will check against the queues from the most recent
reinitialization.
We may have to call getQueueInfo for everytime when config mutation request
comes in, the frequency of mutation request should not super high, I think the
stateless approach should be fine.
bq. This is not implemented yet, but I was thinking of handling this in
RMWebServices, there are some cases that have not been handled (e.g. updating
config for a queue which doesn't exist shouldn't be allowed, right now it
"succeeds" silently). So we can address these cases in a separate jira.
Agree, it will add a never used option, it doesn't sound like a critical issue,
we can handle it in a separate JIRA.
bq. Yes you're right, this is not handled yet, in fact there is still some
handling we need to do in RMWebServices for global configs, we can address this
in a separate jira as well.
If non-trivial effort need to take for this, I'm OK to move it to a separate
JIRA. This is quite important to me. In fact, I think we should not assume any
scheduler-specific configurations inside RMWebServices (like add special logics
to handle "yarn.scheduler.capacity.").
Thoughts?
> Add pluggable configuration policy interface as a component of
> MutableCSConfigurationProvider
> ---------------------------------------------------------------------------------------------
>
> Key: YARN-5949
> URL: https://issues.apache.org/jira/browse/YARN-5949
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Jonathan Hung
> Assignee: Jonathan Hung
> Attachments: YARN-5949-YARN-5734.001.patch,
> YARN-5949-YARN-5734.002.patch
>
>
> This will allow different policies to customize how/if configuration changes
> should be applied (for example, a policy might restrict whether a
> configuration change by a certain user is allowed). This will be enforced by
> the MutableCSConfigurationProvider.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
