[ https://issues.apache.org/jira/browse/YARN-11014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Szilard Nemeth reopened YARN-11014: ----------------------------------- > YARN incorrectly validates maximum capacity resources on the validation API > ----------------------------------------------------------------------------- > > Key: YARN-11014 > URL: https://issues.apache.org/jira/browse/YARN-11014 > Project: Hadoop YARN > Issue Type: Bug > Reporter: Benjamin Teke > Assignee: Benjamin Teke > Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > > The YARN validation API > (http://<host>:8088/ws/v1/cluster/scheduler-conf/validate) allows a > configuration in which a child queue has a greater maximum capacity value > than its parent (the same update fails when it is attempted for real) using > the DominantResourceCalculator. For example, the following passes validation > when the root queue's maximum capacity is less than the 100000000 it is > attempting to set here: > {code:java} > { > "update-queue": [ > { > "queue-name": "root.test", > "params": { > "entry": [ > { > "key": "maximum-capacity", > "value": "[memory=100000000,vcores=3]" > }, > { > "key": "capacity", > "value": "[memory=3020,vcores=1]" > } > ] > } > } > ] > } > {code} > The reason for this is the newly created CapacityScheduler instance doesn't > have the active nodes of the cluster in the nodeTracker object, hence the > clusterResources will be Resources.none() during the init/reinit phase of > this new instance. If the dominantRC has invalid denominator (clusterResource > being 0) it simply compares the two values resource-by-resource: > {code:java} > @Override > public int compare(Resource clusterResource, Resource lhs, Resource rhs, > boolean singleType) { > if (lhs.equals(rhs)) { > return 0; > } > if (isAllInvalidDivisor(clusterResource)) { > return this.compare(lhs, rhs); > } > ... > /** > * Compare two resources - if the value for every resource type for the lhs > * is greater than that of the rhs, return 1. If the value for every > resource > * type in the lhs is less than the rhs, return -1. Otherwise, return 0 > * > * @param lhs resource to be compared > * @param rhs resource to be compared > * @return 0, 1, or -1 > */ > private int compare(Resource lhs, Resource rhs) { > {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org