[
https://issues.apache.org/jira/browse/YARN-9055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16699460#comment-16699460
]
Aihua Xu commented on YARN-9055:
--------------------------------
[~jlowe], [~leftnoteasy] and [~tgraves] Can you guys have any opinions on this?
> Capacity Scheduler: allow larger queue level maximum-allocation-mb to
> override the cluster configuration
> --------------------------------------------------------------------------------------------------------
>
> Key: YARN-9055
> URL: https://issues.apache.org/jira/browse/YARN-9055
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler
> Affects Versions: 2.7.0
> Reporter: Aihua Xu
> Assignee: Aihua Xu
> Priority: Major
>
> YARN-1582 adds the support of maximum-allocation-mb configuration per queue.
> That feature gives the flexibility to give different memory requirements for
> different queues. Such patch adds the limitation that the queue level
> configuration can't exceed the cluster level default configuration, but I
> feel it may make more sense to remove such limitation to allow any overrides
> since
> # Such configuration is controlled by the admin so it shouldn't get abused;
> # It's common that typical queues require standard size containers while some
> job (queues) have requirements for larger containers. With current
> limitation, we have to set larger configuration on the cluster setting which
> will cause resource abuse unless we override them on all the queues.
> We can remove such limitation in CapacitySchedulerConfiguration.java so the
> cluster setting provides the default value and queue setting can override it.
> {noformat}
> if (maxAllocationMbPerQueue > clusterMax.getMemorySize()
> || maxAllocationVcoresPerQueue > clusterMax.getVirtualCores()) {
> throw new IllegalArgumentException(
> "Queue maximum allocation cannot be larger than the cluster setting"
> + " for queue " + queue
> + " max allocation per queue: " + result
> + " cluster setting: " + clusterMax);
> }
> {noformat}
> Let me know if it makes sense.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
