[ https://issues.apache.org/jira/browse/YARN-10287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prabhu Joseph updated YARN-10287: --------------------------------- Attachment: YARN-10287-001.patch > Update scheduler-conf corrupts the CS configuration when removing queue which > is referred in queue mapping > ---------------------------------------------------------------------------------------------------------- > > Key: YARN-10287 > URL: https://issues.apache.org/jira/browse/YARN-10287 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler > Affects Versions: 3.3.0 > Reporter: Akhil PB > Assignee: Prabhu Joseph > Priority: Major > Attachments: YARN-10287-001.patch > > > Update scheduler-conf corrupts the CS configuration when removing queue which > is referred in queue mapping. The deletion is failed with below error > message but the queue got removed from CS configuration and job submission > failed and not removed from the backend ZKConfigurationStore. On subsequent > modify using scheduler-conf, the queue appears again from ZKConfigurationStore > {code} > 2020-05-22 12:38:38,252 ERROR > org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices: Exception > thrown when modifying configuration. > java.io.IOException: Failed to re-init queues : mapping contains invalid or > non-leaf queue Prod > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:478) > at > org.apache.hadoop.yarn.server.resourcemanager.AdminService.refreshQueues(AdminService.java:430) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2389) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2377) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.updateSchedulerConfiguration(RMWebServices.java:2377) > {code} > *Repro:* > {code} > 1. Setup Queue Mapping > yarn.scheduler.capacity.root.queues=default,dummy > yarn.scheduler.capacity.queue-mappings=g:hadoop:dummy > 2. Stop the root.dummy queue > <update-queue> > <queue-name>root.dummy</queue-name> > <params> > <entry> > <key>state</key> > <value>STOPPED</value> > </entry> > </params> > </update-queue> > > > 3. Delete the root.dummy queue > curl --negotiate -u : -X PUT -d @abc.xml -H "Content-type: application/xml" > 'http://<RM_IP>:8088/ws/v1/cluster/scheduler-conf?user.name=yarn' > <sched-conf> > <update-queue> > <queue-name>root.default</queue-name> > <params> > <entry> > <key>capacity</key> > <value>100</value> > </entry> > </params> > </update-queue> > <remove-queue>root.dummy</remove-queue> > </sched-conf> > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org