[
https://issues.apache.org/jira/browse/YARN-11014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved YARN-11014.
-----------------------------------
Fix Version/s: 3.3.3
(was: 3.4.0)
(was: 3.3.4)
Resolution: Fixed
> YARN incorrectly validates maximum capacity resources on the validation API
> -----------------------------------------------------------------------------
>
> Key: YARN-11014
> URL: https://issues.apache.org/jira/browse/YARN-11014
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Benjamin Teke
> Assignee: Benjamin Teke
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.2.4, 3.3.3
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> The YARN validation API
> (http://<host>:8088/ws/v1/cluster/scheduler-conf/validate) allows a
> configuration in which a child queue has a greater maximum capacity value
> than its parent (the same update fails when it is attempted for real) using
> the DominantResourceCalculator.
> For example, the following passes validation when the root queue's maximum
> capacity is less than the 100000000 it is attempting to set here:
> {code:java}
> {
> "update-queue": [
> {
> "queue-name": "root.test",
> "params": {
> "entry": [
> {
> "key": "maximum-capacity",
> "value": "[memory=100000000,vcores=3]"
> },
> {
> "key": "capacity",
> "value": "[memory=3020,vcores=1]"
> }
> ]
> }
> }
> ]
> }
> {code}
> The reason for this is the newly created CapacityScheduler instance doesn't
> have the active nodes of the cluster in the nodeTracker object, hence the
> clusterResources will be Resources.none() during the init/reinit phase of
> this new instance.
> If the dominantRC has invalid denominator (clusterResource being 0) it simply
> compares the two values resource-by-resource:
> {code:java}
> @Override
> public int compare(Resource clusterResource, Resource lhs, Resource rhs,
> boolean singleType) {
> if (lhs.equals(rhs)) {
> return 0;
> }
> if (isAllInvalidDivisor(clusterResource)) {
> return this.compare(lhs, rhs);
> }
> ...
> /**
> * Compare two resources - if the value for every resource type for the lhs
> * is greater than that of the rhs, return 1. If the value for every
> resource
> * type in the lhs is less than the rhs, return -1. Otherwise, return 0
> *
> * @param lhs resource to be compared
> * @param rhs resource to be compared
> * @return 0, 1, or -1
> */
> private int compare(Resource lhs, Resource rhs) {
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
