[jira] [Updated] (YARN-9925) CapacitySchedulerQueueManager allows unsupported Queue hierarchy
[ https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9925: Release Note: Currently, CapacityScheduler does not allow two queues with same name irrespective of hierarchy or position. For example, root.a.a1 and root.b.a1 are not allowed. root.a.b.c and root.a.d.b.e are not allowed. > CapacitySchedulerQueueManager allows unsupported Queue hierarchy > > > Key: YARN-9925 > URL: https://issues.apache.org/jira/browse/YARN-9925 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: YARN-9925-001.patch, YARN-9925-002.patch, > YARN-9925-003.patch, YARN-9925-004.patch, YARN-9925-005.patch > > > CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When > creating a queue with same name as an existing parent queue name - it has to > fail with below. > {code:java} > Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after > refresh, which is not allowed.Caused by: java.io.IOException: A is moved > from:root.A to:root.B.A after refresh, which is not allowed. at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) > ... 70 more > {code} > In Some cases, the error is not thrown while creating the queue but thrown at > submission of job "Failed to submit application_1571677375269_0002 to YARN : > Application application_1571677375269_0002 submitted by user : systest to > non-leaf queue : B" > Below scenarios are allowed but it should not > {code:java} > It allows root.A.A1.B when root.B.B1 already exists. > > 1. Add root.A > 2. Add root.A.A1 > 3. Add root.B > 4. Add root.B.B1 > 5. Allows Add of root.A.A1.B > It allows two root queues: > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Allows Add of root.A.A1.root > > {code} > Below scenario is handled properly: > {code:java} > It does not allow root.B.A when root.A.A1 already exists. > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Does not Allow Add of root.B.A > {code} > This error handling has to be consistent in all scenarios. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9925) CapacitySchedulerQueueManager allows unsupported Queue hierarchy
[ https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9925: Attachment: YARN-9925-005.patch > CapacitySchedulerQueueManager allows unsupported Queue hierarchy > > > Key: YARN-9925 > URL: https://issues.apache.org/jira/browse/YARN-9925 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: YARN-9925-001.patch, YARN-9925-002.patch, > YARN-9925-003.patch, YARN-9925-004.patch, YARN-9925-005.patch > > > CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When > creating a queue with same name as an existing parent queue name - it has to > fail with below. > {code:java} > Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after > refresh, which is not allowed.Caused by: java.io.IOException: A is moved > from:root.A to:root.B.A after refresh, which is not allowed. at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) > ... 70 more > {code} > In Some cases, the error is not thrown while creating the queue but thrown at > submission of job "Failed to submit application_1571677375269_0002 to YARN : > Application application_1571677375269_0002 submitted by user : systest to > non-leaf queue : B" > Below scenarios are allowed but it should not > {code:java} > It allows root.A.A1.B when root.B.B1 already exists. > > 1. Add root.A > 2. Add root.A.A1 > 3. Add root.B > 4. Add root.B.B1 > 5. Allows Add of root.A.A1.B > It allows two root queues: > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Allows Add of root.A.A1.root > > {code} > Below scenario is handled properly: > {code:java} > It does not allow root.B.A when root.A.A1 already exists. > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Does not Allow Add of root.B.A > {code} > This error handling has to be consistent in all scenarios. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9925) CapacitySchedulerQueueManager allows unsupported Queue hierarchy
[ https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9925: Attachment: YARN-9925-004.patch > CapacitySchedulerQueueManager allows unsupported Queue hierarchy > > > Key: YARN-9925 > URL: https://issues.apache.org/jira/browse/YARN-9925 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: YARN-9925-001.patch, YARN-9925-002.patch, > YARN-9925-003.patch, YARN-9925-004.patch > > > CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When > creating a queue with same name as an existing parent queue name - it has to > fail with below. > {code:java} > Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after > refresh, which is not allowed.Caused by: java.io.IOException: A is moved > from:root.A to:root.B.A after refresh, which is not allowed. at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) > ... 70 more > {code} > In Some cases, the error is not thrown while creating the queue but thrown at > submission of job "Failed to submit application_1571677375269_0002 to YARN : > Application application_1571677375269_0002 submitted by user : systest to > non-leaf queue : B" > Below scenarios are allowed but it should not > {code:java} > It allows root.A.A1.B when root.B.B1 already exists. > > 1. Add root.A > 2. Add root.A.A1 > 3. Add root.B > 4. Add root.B.B1 > 5. Allows Add of root.A.A1.B > It allows two root queues: > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Allows Add of root.A.A1.root > > {code} > Below scenario is handled properly: > {code:java} > It does not allow root.B.A when root.A.A1 already exists. > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Does not Allow Add of root.B.A > {code} > This error handling has to be consistent in all scenarios. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9925) CapacitySchedulerQueueManager allows unsupported Queue hierarchy
[ https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9925: Attachment: YARN-9925-003.patch > CapacitySchedulerQueueManager allows unsupported Queue hierarchy > > > Key: YARN-9925 > URL: https://issues.apache.org/jira/browse/YARN-9925 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: YARN-9925-001.patch, YARN-9925-002.patch, > YARN-9925-003.patch > > > CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When > creating a queue with same name as an existing parent queue name - it has to > fail with below. > {code:java} > Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after > refresh, which is not allowed.Caused by: java.io.IOException: A is moved > from:root.A to:root.B.A after refresh, which is not allowed. at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) > ... 70 more > {code} > In Some cases, the error is not thrown while creating the queue but thrown at > submission of job "Failed to submit application_1571677375269_0002 to YARN : > Application application_1571677375269_0002 submitted by user : systest to > non-leaf queue : B" > Below scenarios are allowed but it should not > {code:java} > It allows root.A.A1.B when root.B.B1 already exists. > > 1. Add root.A > 2. Add root.A.A1 > 3. Add root.B > 4. Add root.B.B1 > 5. Allows Add of root.A.A1.B > It allows two root queues: > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Allows Add of root.A.A1.root > > {code} > Below scenario is handled properly: > {code:java} > It does not allow root.B.A when root.A.A1 already exists. > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Does not Allow Add of root.B.A > {code} > This error handling has to be consistent in all scenarios. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9925) CapacitySchedulerQueueManager allows unsupported Queue hierarchy
[ https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9925: Attachment: (was: YARN-9925-003.patch) > CapacitySchedulerQueueManager allows unsupported Queue hierarchy > > > Key: YARN-9925 > URL: https://issues.apache.org/jira/browse/YARN-9925 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: YARN-9925-001.patch, YARN-9925-002.patch > > > CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When > creating a queue with same name as an existing parent queue name - it has to > fail with below. > {code:java} > Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after > refresh, which is not allowed.Caused by: java.io.IOException: A is moved > from:root.A to:root.B.A after refresh, which is not allowed. at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) > ... 70 more > {code} > In Some cases, the error is not thrown while creating the queue but thrown at > submission of job "Failed to submit application_1571677375269_0002 to YARN : > Application application_1571677375269_0002 submitted by user : systest to > non-leaf queue : B" > Below scenarios are allowed but it should not > {code:java} > It allows root.A.A1.B when root.B.B1 already exists. > > 1. Add root.A > 2. Add root.A.A1 > 3. Add root.B > 4. Add root.B.B1 > 5. Allows Add of root.A.A1.B > It allows two root queues: > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Allows Add of root.A.A1.root > > {code} > Below scenario is handled properly: > {code:java} > It does not allow root.B.A when root.A.A1 already exists. > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Does not Allow Add of root.B.A > {code} > This error handling has to be consistent in all scenarios. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9925) CapacitySchedulerQueueManager allows unsupported Queue hierarchy
[ https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9925: Attachment: YARN-9925-003.patch > CapacitySchedulerQueueManager allows unsupported Queue hierarchy > > > Key: YARN-9925 > URL: https://issues.apache.org/jira/browse/YARN-9925 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: YARN-9925-001.patch, YARN-9925-002.patch, > YARN-9925-003.patch > > > CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When > creating a queue with same name as an existing parent queue name - it has to > fail with below. > {code:java} > Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after > refresh, which is not allowed.Caused by: java.io.IOException: A is moved > from:root.A to:root.B.A after refresh, which is not allowed. at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) > ... 70 more > {code} > In Some cases, the error is not thrown while creating the queue but thrown at > submission of job "Failed to submit application_1571677375269_0002 to YARN : > Application application_1571677375269_0002 submitted by user : systest to > non-leaf queue : B" > Below scenarios are allowed but it should not > {code:java} > It allows root.A.A1.B when root.B.B1 already exists. > > 1. Add root.A > 2. Add root.A.A1 > 3. Add root.B > 4. Add root.B.B1 > 5. Allows Add of root.A.A1.B > It allows two root queues: > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Allows Add of root.A.A1.root > > {code} > Below scenario is handled properly: > {code:java} > It does not allow root.B.A when root.A.A1 already exists. > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Does not Allow Add of root.B.A > {code} > This error handling has to be consistent in all scenarios. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9925) CapacitySchedulerQueueManager allows unsupported Queue hierarchy
[ https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9925: Attachment: (was: YARN-9925-003.patch) > CapacitySchedulerQueueManager allows unsupported Queue hierarchy > > > Key: YARN-9925 > URL: https://issues.apache.org/jira/browse/YARN-9925 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: YARN-9925-001.patch, YARN-9925-002.patch > > > CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When > creating a queue with same name as an existing parent queue name - it has to > fail with below. > {code:java} > Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after > refresh, which is not allowed.Caused by: java.io.IOException: A is moved > from:root.A to:root.B.A after refresh, which is not allowed. at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) > ... 70 more > {code} > In Some cases, the error is not thrown while creating the queue but thrown at > submission of job "Failed to submit application_1571677375269_0002 to YARN : > Application application_1571677375269_0002 submitted by user : systest to > non-leaf queue : B" > Below scenarios are allowed but it should not > {code:java} > It allows root.A.A1.B when root.B.B1 already exists. > > 1. Add root.A > 2. Add root.A.A1 > 3. Add root.B > 4. Add root.B.B1 > 5. Allows Add of root.A.A1.B > It allows two root queues: > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Allows Add of root.A.A1.root > > {code} > Below scenario is handled properly: > {code:java} > It does not allow root.B.A when root.A.A1 already exists. > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Does not Allow Add of root.B.A > {code} > This error handling has to be consistent in all scenarios. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9925) CapacitySchedulerQueueManager allows unsupported Queue hierarchy
[ https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9925: Attachment: YARN-9925-003.patch > CapacitySchedulerQueueManager allows unsupported Queue hierarchy > > > Key: YARN-9925 > URL: https://issues.apache.org/jira/browse/YARN-9925 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: YARN-9925-001.patch, YARN-9925-002.patch, > YARN-9925-003.patch > > > CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When > creating a queue with same name as an existing parent queue name - it has to > fail with below. > {code:java} > Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after > refresh, which is not allowed.Caused by: java.io.IOException: A is moved > from:root.A to:root.B.A after refresh, which is not allowed. at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) > ... 70 more > {code} > In Some cases, the error is not thrown while creating the queue but thrown at > submission of job "Failed to submit application_1571677375269_0002 to YARN : > Application application_1571677375269_0002 submitted by user : systest to > non-leaf queue : B" > Below scenarios are allowed but it should not > {code:java} > It allows root.A.A1.B when root.B.B1 already exists. > > 1. Add root.A > 2. Add root.A.A1 > 3. Add root.B > 4. Add root.B.B1 > 5. Allows Add of root.A.A1.B > It allows two root queues: > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Allows Add of root.A.A1.root > > {code} > Below scenario is handled properly: > {code:java} > It does not allow root.B.A when root.A.A1 already exists. > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Does not Allow Add of root.B.A > {code} > This error handling has to be consistent in all scenarios. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9925) CapacitySchedulerQueueManager allows unsupported Queue hierarchy
[ https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9925: Attachment: YARN-9925-002.patch > CapacitySchedulerQueueManager allows unsupported Queue hierarchy > > > Key: YARN-9925 > URL: https://issues.apache.org/jira/browse/YARN-9925 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: YARN-9925-001.patch, YARN-9925-002.patch > > > CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When > creating a queue with same name as an existing parent queue name - it has to > fail with below. > {code:java} > Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after > refresh, which is not allowed.Caused by: java.io.IOException: A is moved > from:root.A to:root.B.A after refresh, which is not allowed. at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) > ... 70 more > {code} > In Some cases, the error is not thrown while creating the queue but thrown at > submission of job "Failed to submit application_1571677375269_0002 to YARN : > Application application_1571677375269_0002 submitted by user : systest to > non-leaf queue : B" > Below scenarios are allowed but it should not > {code:java} > It allows root.A.A1.B when root.B.B1 already exists. > > 1. Add root.A > 2. Add root.A.A1 > 3. Add root.B > 4. Add root.B.B1 > 5. Allows Add of root.A.A1.B > It allows two root queues: > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Allows Add of root.A.A1.root > > {code} > Below scenario is handled properly: > {code:java} > It does not allow root.B.A when root.A.A1 already exists. > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Does not Allow Add of root.B.A > {code} > This error handling has to be consistent in all scenarios. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9925) CapacitySchedulerQueueManager allows unsupported Queue hierarchy
[ https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9925: Attachment: YARN-9925-001.patch > CapacitySchedulerQueueManager allows unsupported Queue hierarchy > > > Key: YARN-9925 > URL: https://issues.apache.org/jira/browse/YARN-9925 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: YARN-9925-001.patch > > > CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When > creating a queue with same name as an existing parent queue name - it has to > fail with below. > {code:java} > Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after > refresh, which is not allowed.Caused by: java.io.IOException: A is moved > from:root.A to:root.B.A after refresh, which is not allowed. at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) > ... 70 more > {code} > In Some cases, the error is not thrown while creating the queue but thrown at > submission of job "Failed to submit application_1571677375269_0002 to YARN : > Application application_1571677375269_0002 submitted by user : systest to > non-leaf queue : B" > Below scenarios are allowed but it should not > {code:java} > It allows root.A.A1.B when root.B.B1 already exists. > > 1. Add root.A > 2. Add root.A.A1 > 3. Add root.B > 4. Add root.B.B1 > 5. Allows Add of root.A.A1.B > It allows two root queues: > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Allows Add of root.A.A1.root > > {code} > Below scenario is handled properly: > {code:java} > It does not allow root.B.A when root.A.A1 already exists. > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Does not Allow Add of root.B.A > {code} > This error handling has to be consistent in all scenarios. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9925) CapacitySchedulerQueueManager allows unsupported Queue hierarchy
[ https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9925: Description: CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When creating a queue with same name as an existing parent queue name - it has to fail with below. {code:java} Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after refresh, which is not allowed.Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after refresh, which is not allowed. at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) ... 70 more {code} In Some cases, the error is not thrown while creating the queue but thrown at submission of job "Failed to submit application_1571677375269_0002 to YARN : Application application_1571677375269_0002 submitted by user : systest to non-leaf queue : B" Below scenarios are allowed but it should not {code:java} It allows root.A.A1.B when root.B.B1 already exists. 1. Add root.A 2. Add root.A.A1 3. Add root.B 4. Add root.B.B1 5. Allows Add of root.A.A1.B It allows two root queues: 1. Add root.A 2. Add root.B 3. Add root.A.A1 4. Allows Add of root.A.A1.root {code} Below scenario is handled properly: {code:java} It does not allow root.B.A when root.A.A1 already exists. 1. Add root.A 2. Add root.B 3. Add root.A.A1 4. Does not Allow Add of root.B.A {code} This error handling has to be consistent in all scenarios. was: CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When creating a queue with same name as an existing parent queue name - it has to fail with below. {code:java} Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after refresh, which is not allowed.Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after refresh, which is not allowed. at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) ... 70 more {code} In Some cases, the error is not thrown while creating the queue but thrown at submission of job "Failed to submit application_1571677375269_0002 to YARN : Application application_1571677375269_0002 submitted by user : systest to non-leaf queue : B" Below scenarios are allowed but it should not {code:java} It allows root.A.A1.B when root.B.B1 exists already. 1. Add root.A 2. Add root.A.A1 3. Add root.B 4. Add root.B.B1 5. Allows Add of root.A.A1.B It allows two root queues: 1. Add root.A 2. Add root.B 3. Add root.A.A1 4. Allows Add of root.A.A1.root {code} Below scenario is handled properly: {code:java} It does not allow root.B.A when root.A.A1 exists already. 1. Add root.A 2. Add root.B 3. Add root.A.A1 4. Does not Allow Add of root.B.A {code} This error handling has to be consistent in all scenarios. > CapacitySchedulerQueueManager allows unsupported Queue hierarchy > > > Key: YARN-9925 > URL: https://issues.apache.org/jira/browse/YARN-9925 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > > CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When > creating a queue with same name as an existing parent queue name - it has to > fail with below. > {code:java} > Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after > refresh, which is not allowed.Caused by: java.io.IOException: A is moved > from:root.A to:root.B.A after refresh, which is not allowed. at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) > at >
[jira] [Updated] (YARN-9925) CapacitySchedulerQueueManager allows unsupported Queue hierarchy
[ https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9925: Affects Version/s: 3.3.0 > CapacitySchedulerQueueManager allows unsupported Queue hierarchy > > > Key: YARN-9925 > URL: https://issues.apache.org/jira/browse/YARN-9925 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > > CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When > creating a queue with same name as an existing parent queue name - it has to > fail with below. > {code:java} > Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after > refresh, which is not allowed.Caused by: java.io.IOException: A is moved > from:root.A to:root.B.A after refresh, which is not allowed. at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) > ... 70 more > {code} > In Some cases, the error is not thrown while creating the queue but thrown at > submission of job "Failed to submit application_1571677375269_0002 to YARN : > Application application_1571677375269_0002 submitted by user : systest to > non-leaf queue : B" > Below scenarios are allowed but it should not > {code:java} > It allows root.A.A1.B when root.B.B1 exists already. > > 1. Add root.A > 2. Add root.A.A1 > 3. Add root.B > 4. Add root.B.B1 > 5. Allows Add of root.A.A1.B > It allows two root queues: > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Allows Add of root.A.A1.root > > {code} > Below scenario is handled properly: > {code:java} > It does not allow root.B.A when root.A.A1 exists already. > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Does not Allow Add of root.B.A > {code} > This error handling has to be consistent in all scenarios. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9925) CapacitySchedulerQueueManager allows unsupported Queue hierarchy
[ https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9925: Component/s: capacityscheduler > CapacitySchedulerQueueManager allows unsupported Queue hierarchy > > > Key: YARN-9925 > URL: https://issues.apache.org/jira/browse/YARN-9925 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > > CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When > creating a queue with same name as an existing parent queue name - it has to > fail with below. > {code:java} > Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after > refresh, which is not allowed.Caused by: java.io.IOException: A is moved > from:root.A to:root.B.A after refresh, which is not allowed. at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) > ... 70 more > {code} > In Some cases, the error is not thrown while creating the queue but thrown at > submission of job "Failed to submit application_1571677375269_0002 to YARN : > Application application_1571677375269_0002 submitted by user : systest to > non-leaf queue : B" > Below scenarios are allowed but it should not > {code:java} > It allows root.A.A1.B when root.B.B1 exists already. > > 1. Add root.A > 2. Add root.A.A1 > 3. Add root.B > 4. Add root.B.B1 > 5. Allows Add of root.A.A1.B > It allows two root queues: > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Allows Add of root.A.A1.root > > {code} > Below scenario is handled properly: > {code:java} > It does not allow root.B.A when root.A.A1 exists already. > > 1. Add root.A > 2. Add root.B > 3. Add root.A.A1 > 4. Does not Allow Add of root.B.A > {code} > This error handling has to be consistent in all scenarios. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9925) CapacitySchedulerQueueManager allows unsupported Queue hierarchy
[ https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9925: Description: CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When creating a queue with same name as an existing parent queue name - it has to fail with below. {code:java} Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after refresh, which is not allowed.Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after refresh, which is not allowed. at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) ... 70 more {code} In Some cases, the error is not thrown while creating the queue but thrown at submission of job "Failed to submit application_1571677375269_0002 to YARN : Application application_1571677375269_0002 submitted by user : systest to non-leaf queue : B" Below scenarios are allowed but it should not {code:java} It allows root.A.A1.B when root.B.B1 exists already. 1. Add root.A 2. Add root.A.A1 3. Add root.B 4. Add root.B.B1 5. Allows Add of root.A.A1.B It allows two root queues: 1. Add root.A 2. Add root.B 3. Add root.A.A1 4. Allows Add of root.A.A1.root {code} Below scenario is handled properly: {code:java} It does not allow root.B.A when root.A.A1 exists already. 1. Add root.A 2. Add root.B 3. Add root.A.A1 4. Does not Allow Add of root.B.A {code} This error handling has to be consistent in all scenarios. was: CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When creating a queue with same name as an existing parent queue name - it has to fail with below. {code:java} Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after refresh, which is not allowed.Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after refresh, which is not allowed. at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) ... 70 more {code} In Some cases, the error is not thrown while creating the queue but thrown at submission of job "Failed to submit application_1571677375269_0002 to YARN : Application application_1571677375269_0002 submitted by user : systest to non-leaf queue : B" Below scenarios are allowed but it should not {code} 1. It allows root.A.A1.B when root.B.B1 exists already. 1. Add root.A 2. Add root.A.A1 3. Add root.B 4. Add root.B.B1 5. Allows Add of root.A.A1.B 2. It allows two root queues: 1. Add root.A 2. Add root.B 3. Add root.A.A1 4. Allows Add of root.A.A1.root {code} Below scenario is handled properly: {code} 1. It does not allow root.B.A when root.A.A1 exists already. 1. Add root.A 2. Add root.B 3. Add root.A.A1 4. Does not Allow Add of root.B.A {code} This error handling has to be consistent in all scenarios. > CapacitySchedulerQueueManager allows unsupported Queue hierarchy > > > Key: YARN-9925 > URL: https://issues.apache.org/jira/browse/YARN-9925 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > > CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When > creating a queue with same name as an existing parent queue name - it has to > fail with below. > {code:java} > Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after > refresh, which is not allowed.Caused by: java.io.IOException: A is moved > from:root.A to:root.B.A after refresh, which is not allowed. at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) > at >