[jira] [Commented] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[ https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405546#comment-16405546 ] Yufei Gu commented on YARN-4022: Attached the design document. > queue not remove from webpage(/cluster/scheduler) when delete queue in > xxx-scheduler.xml > > > Key: YARN-4022 > URL: https://issues.apache.org/jira/browse/YARN-4022 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 2.7.1 >Reporter: forrestchen >Assignee: Szilard Nemeth >Priority: Major > Labels: oct16-medium, scheduler > Attachments: Queue Deletion in Fair Scheduler.pdf, > YARN-4022.001.patch, YARN-4022.002.patch, YARN-4022.003.patch, > YARN-4022.004.patch > > > When I delete an existing queue by modify the xxx-schedule.xml, I can still > see the queue information block in webpage(/cluster/scheduler) though the > 'Min Resources' items all become to zero and have no item of 'Max Running > Applications'. > I can still submit an application to the deleted queue and the application > will run using 'root.default' queue instead, but submit to an un-exist queue > will cause an exception. > My expectation is the deleted queue will not displayed in webpage and submit > application to the deleted queue will act just like the queue doesn't exist. > PS: There's no application running in the queue I delete. > Some related config in yarn-site.xml: > {code} > > yarn.scheduler.fair.user-as-default-queue > false > > > yarn.scheduler.fair.allow-undeclared-pools > false > > {code} > a related question is here: > http://stackoverflow.com/questions/26488564/hadoop-yarn-why-the-queue-cannot-be-deleted-after-i-revise-my-fair-scheduler-xm -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[ https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337412#comment-16337412 ] Szilard Nemeth commented on YARN-4022: -- Hey [~yufeigu]! Sorry, there was a confusion from my side so please ignore my last comment, especially a part about -refreshQueues since this bug is about the allocations.xml file, as I see it. > queue not remove from webpage(/cluster/scheduler) when delete queue in > xxx-scheduler.xml > > > Key: YARN-4022 > URL: https://issues.apache.org/jira/browse/YARN-4022 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 2.7.1 >Reporter: forrestchen >Assignee: Szilard Nemeth >Priority: Major > Labels: oct16-medium, scheduler > Attachments: YARN-4022.001.patch, YARN-4022.002.patch, > YARN-4022.003.patch, YARN-4022.004.patch > > > When I delete an existing queue by modify the xxx-schedule.xml, I can still > see the queue information block in webpage(/cluster/scheduler) though the > 'Min Resources' items all become to zero and have no item of 'Max Running > Applications'. > I can still submit an application to the deleted queue and the application > will run using 'root.default' queue instead, but submit to an un-exist queue > will cause an exception. > My expectation is the deleted queue will not displayed in webpage and submit > application to the deleted queue will act just like the queue doesn't exist. > PS: There's no application running in the queue I delete. > Some related config in yarn-site.xml: > {code} > > yarn.scheduler.fair.user-as-default-queue > false > > > yarn.scheduler.fair.allow-undeclared-pools > false > > {code} > a related question is here: > http://stackoverflow.com/questions/26488564/hadoop-yarn-why-the-queue-cannot-be-deleted-after-i-revise-my-fair-scheduler-xm -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[ https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336310#comment-16336310 ] Yufei Gu commented on YARN-4022: For your questions: # Need further investigation. # No. We shouldn't do that. # A dynamic queue gets a parent who is labeled as "parent" and its queue type is "LEAF" # I don't think this Jira is to implement functionality of deleting queues. I believe you want to do something similar to YARN-5556. However, I think the scope of this Jira is limited, which aims to refresh the RM WebUI while it shouldn't show the queue if the queue has been deleted by RM. It is debatable whether we need the similar functionality in YARN-5556, and I'd prefer to file a new Jira for that and move the discussion to there. > queue not remove from webpage(/cluster/scheduler) when delete queue in > xxx-scheduler.xml > > > Key: YARN-4022 > URL: https://issues.apache.org/jira/browse/YARN-4022 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 2.7.1 >Reporter: forrestchen >Assignee: Szilard Nemeth >Priority: Major > Labels: oct16-medium, scheduler > Attachments: YARN-4022.001.patch, YARN-4022.002.patch, > YARN-4022.003.patch, YARN-4022.004.patch > > > When I delete an existing queue by modify the xxx-schedule.xml, I can still > see the queue information block in webpage(/cluster/scheduler) though the > 'Min Resources' items all become to zero and have no item of 'Max Running > Applications'. > I can still submit an application to the deleted queue and the application > will run using 'root.default' queue instead, but submit to an un-exist queue > will cause an exception. > My expectation is the deleted queue will not displayed in webpage and submit > application to the deleted queue will act just like the queue doesn't exist. > PS: There's no application running in the queue I delete. > Some related config in yarn-site.xml: > {code} > > yarn.scheduler.fair.user-as-default-queue > false > > > yarn.scheduler.fair.allow-undeclared-pools > false > > {code} > a related question is here: > http://stackoverflow.com/questions/26488564/hadoop-yarn-why-the-queue-cannot-be-deleted-after-i-revise-my-fair-scheduler-xm -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[ https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334116#comment-16334116 ] Szilard Nemeth commented on YARN-4022: -- Hey @danieltempleton, @yufei! Could you please help me a bit with this one? I left the _{{yarn.scheduler.fair.user-as-default-queue}}_ and _{{yarn.scheduler.fair.allow-undeclared-pools}}_ configs on their default values (true) as based on the FairScheduler page ([https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html]), this makes the most sense to have dynamic queues created. When I started a pi job, the job was assigned to a dynamically created queue, namely "root.szilardnemeth" so I guess the above config is correct. When I ran {{yarn rmadmin -refreshQueues}} and checked the RM Webservices API with a GET request to "ws/v1/cluster/scheduler", the queue was still there. After that, I debugged the calls described below and found out that the queues are not deleted when refreshQueues is invoked even if they are empty. Eventually, {{org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager#removeEmptyIncompatibleQueues}} is invoked but this method does not delete my dynamically created leaf queue, moreover this method does not seem to be a good fit to add queue removal functionality, since it only deals with incompatible queues. I found out that when I start the command {{yarn rmadmin -refreshQueues}}, the following relevant calls are performed: 1. {{AdminService.refreshQueues}} handles the CLI command 2. {{org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler#reinitialize}} is invoked 3. The method above invokes {{org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService#reloadAllocations}} which loads the allocations.xml file. At the end of this method, a call happens to the {{reloadListener}} with the parsed configuration object: {{reloadListener.onReload(info);}} 4. {{org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.AllocationReloadListener#onReload}} is invoked. 5. The method above invokes {{org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager#updateAllocationConfiguration}} This method is responsible for removing incompatible queues, see {{removeEmptyIncompatibleQueues}} in {{QueueManager}} (at the time of writing: [https://github.com/apache/hadoop/blob/99292adcefdc6b8f280b8e100605fb39f755c38a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java#L351]) *For me, adding the queue removal functionality to FairScheduler.reinitialize would be the most logical thing to do, as the rest of the methods are strongly related to reading the allocations file and since dynamically created queues are not based on that file, it is a "separate entity". * My questions: 1. Should all empty dynamically created queues be removed when the refreshQueues command is invoked with the CLI? 2. May all empty queues be removed when refreshQueues command is invoked or just the dynamically created ones? 3. If the answer is "just the dynamically created queues can be removed" for question 2, how can I differentiate the normal queues from the dynamically created queues? > queue not remove from webpage(/cluster/scheduler) when delete queue in > xxx-scheduler.xml > > > Key: YARN-4022 > URL: https://issues.apache.org/jira/browse/YARN-4022 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 2.7.1 >Reporter: forrestchen >Assignee: Szilard Nemeth >Priority: Major > Labels: oct16-medium, scheduler > Attachments: YARN-4022.001.patch, YARN-4022.002.patch, > YARN-4022.003.patch, YARN-4022.004.patch > > > When I delete an existing queue by modify the xxx-schedule.xml, I can still > see the queue information block in webpage(/cluster/scheduler) though the > 'Min Resources' items all become to zero and have no item of 'Max Running > Applications'. > I can still submit an application to the deleted queue and the application > will run using 'root.default' queue instead, but submit to an un-exist queue > will cause an exception. > My expectation is the deleted queue will not displayed in webpage and submit > application to the deleted queue will act just like the queue doesn't exist. > PS: There's no application running in the queue I delete. > Some related config in yarn-site.xml: > {code} > > yarn.scheduler.fair.user-as-default-queue > false > > > yarn.scheduler.fair.allow-undeclared-pools >
[jira] [Commented] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[ https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849161#comment-15849161 ] Ashwin Shankar commented on YARN-4022: -- [~templedf],[~forrestchen],[~yufeigu], Whoever is planning to work on this patch, please account for 'dynamic user queues' in fair scheduler. These queues don't exist in the xml, but are created on the fly based on username when a user submits a job. Please see fair scheduler docs for more information. So basically queues can exist in the scheduler but not in xml, and shouldn't be deleted just because they don't exist in the xml. > queue not remove from webpage(/cluster/scheduler) when delete queue in > xxx-scheduler.xml > > > Key: YARN-4022 > URL: https://issues.apache.org/jira/browse/YARN-4022 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 2.7.1 >Reporter: forrestchen > Labels: oct16-medium, scheduler > Attachments: YARN-4022.001.patch, YARN-4022.002.patch, > YARN-4022.003.patch, YARN-4022.004.patch > > > When I delete an existing queue by modify the xxx-schedule.xml, I can still > see the queue information block in webpage(/cluster/scheduler) though the > 'Min Resources' items all become to zero and have no item of 'Max Running > Applications'. > I can still submit an application to the deleted queue and the application > will run using 'root.default' queue instead, but submit to an un-exist queue > will cause an exception. > My expectation is the deleted queue will not displayed in webpage and submit > application to the deleted queue will act just like the queue doesn't exist. > PS: There's no application running in the queue I delete. > Some related config in yarn-site.xml: > {code} > > yarn.scheduler.fair.user-as-default-queue > false > > > yarn.scheduler.fair.allow-undeclared-pools > false > > {code} > a related question is here: > http://stackoverflow.com/questions/26488564/hadoop-yarn-why-the-queue-cannot-be-deleted-after-i-revise-my-fair-scheduler-xm -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[ https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812907#comment-15812907 ] Yufei Gu commented on YARN-4022: Hi [~forrestchen], are you still work on this? If not, can I take it? > queue not remove from webpage(/cluster/scheduler) when delete queue in > xxx-scheduler.xml > > > Key: YARN-4022 > URL: https://issues.apache.org/jira/browse/YARN-4022 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 2.7.1 >Reporter: forrestchen > Labels: oct16-medium, scheduler > Attachments: YARN-4022.001.patch, YARN-4022.002.patch, > YARN-4022.003.patch, YARN-4022.004.patch > > > When I delete an existing queue by modify the xxx-schedule.xml, I can still > see the queue information block in webpage(/cluster/scheduler) though the > 'Min Resources' items all become to zero and have no item of 'Max Running > Applications'. > I can still submit an application to the deleted queue and the application > will run using 'root.default' queue instead, but submit to an un-exist queue > will cause an exception. > My expectation is the deleted queue will not displayed in webpage and submit > application to the deleted queue will act just like the queue doesn't exist. > PS: There's no application running in the queue I delete. > Some related config in yarn-site.xml: > {code} > > yarn.scheduler.fair.user-as-default-queue > false > > > yarn.scheduler.fair.allow-undeclared-pools > false > > {code} > a related question is here: > http://stackoverflow.com/questions/26488564/hadoop-yarn-why-the-queue-cannot-be-deleted-after-i-revise-my-fair-scheduler-xm -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[ https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613638#comment-15613638 ] Daniel Templeton commented on YARN-4022: Thanks for the patch, [~forrestchen]. This is definitely useful functionality. It the patch you're using a thread that watches for updates to the configuration so it can remove unused queues. YARN-5556 is adding the same capability for fair scheduler, and that patch handles the delete during the reload of the configuration. Think that approach could work for fair scheduler? I'd also like to see some unit tests to cover the new functionality. > queue not remove from webpage(/cluster/scheduler) when delete queue in > xxx-scheduler.xml > > > Key: YARN-4022 > URL: https://issues.apache.org/jira/browse/YARN-4022 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 2.7.1 >Reporter: forrestchen > Labels: oct16-medium, scheduler > Attachments: YARN-4022.001.patch, YARN-4022.002.patch, > YARN-4022.003.patch, YARN-4022.004.patch > > > When I delete an existing queue by modify the xxx-schedule.xml, I can still > see the queue information block in webpage(/cluster/scheduler) though the > 'Min Resources' items all become to zero and have no item of 'Max Running > Applications'. > I can still submit an application to the deleted queue and the application > will run using 'root.default' queue instead, but submit to an un-exist queue > will cause an exception. > My expectation is the deleted queue will not displayed in webpage and submit > application to the deleted queue will act just like the queue doesn't exist. > PS: There's no application running in the queue I delete. > Some related config in yarn-site.xml: > {code} > > yarn.scheduler.fair.user-as-default-queue > false > > > yarn.scheduler.fair.allow-undeclared-pools > false > > {code} > a related question is here: > http://stackoverflow.com/questions/26488564/hadoop-yarn-why-the-queue-cannot-be-deleted-after-i-revise-my-fair-scheduler-xm -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[ https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15032978#comment-15032978 ] stefanlee commented on YARN-4022: - hi forrestchen, my configuration in cluster is similar to you ,and when i delete a queue in fair-scheduler.xml,I submit an application to the deleted queue and the application will run using 'root.default' queue instead, then submit to an un-exist queue ,but still run using 'root.default' and no exception. why ? my hadoop version is 2.4.0 > queue not remove from webpage(/cluster/scheduler) when delete queue in > xxx-scheduler.xml > > > Key: YARN-4022 > URL: https://issues.apache.org/jira/browse/YARN-4022 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.7.1 >Reporter: forrestchen > Labels: scheduler > Attachments: YARN-4022.001.patch, YARN-4022.002.patch, > YARN-4022.003.patch, YARN-4022.004.patch > > > When I delete an existing queue by modify the xxx-schedule.xml, I can still > see the queue information block in webpage(/cluster/scheduler) though the > 'Min Resources' items all become to zero and have no item of 'Max Running > Applications'. > I can still submit an application to the deleted queue and the application > will run using 'root.default' queue instead, but submit to an un-exist queue > will cause an exception. > My expectation is the deleted queue will not displayed in webpage and submit > application to the deleted queue will act just like the queue doesn't exist. > PS: There's no application running in the queue I delete. > Some related config in yarn-site.xml: > {code} > > yarn.scheduler.fair.user-as-default-queue > false > > > yarn.scheduler.fair.allow-undeclared-pools > false > > {code} > a related question is here: > http://stackoverflow.com/questions/26488564/hadoop-yarn-why-the-queue-cannot-be-deleted-after-i-revise-my-fair-scheduler-xm -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[ https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15033006#comment-15033006 ] Hadoop QA commented on YARN-4022: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color} | {color:red} YARN-4022 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12754857/YARN-4022.004.patch | | JIRA Issue | YARN-4022 | | Powered by | Apache Yetus http://yetus.apache.org | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/9819/console | This message was automatically generated. > queue not remove from webpage(/cluster/scheduler) when delete queue in > xxx-scheduler.xml > > > Key: YARN-4022 > URL: https://issues.apache.org/jira/browse/YARN-4022 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.7.1 >Reporter: forrestchen > Labels: scheduler > Attachments: YARN-4022.001.patch, YARN-4022.002.patch, > YARN-4022.003.patch, YARN-4022.004.patch > > > When I delete an existing queue by modify the xxx-schedule.xml, I can still > see the queue information block in webpage(/cluster/scheduler) though the > 'Min Resources' items all become to zero and have no item of 'Max Running > Applications'. > I can still submit an application to the deleted queue and the application > will run using 'root.default' queue instead, but submit to an un-exist queue > will cause an exception. > My expectation is the deleted queue will not displayed in webpage and submit > application to the deleted queue will act just like the queue doesn't exist. > PS: There's no application running in the queue I delete. > Some related config in yarn-site.xml: > {code} > > yarn.scheduler.fair.user-as-default-queue > false > > > yarn.scheduler.fair.allow-undeclared-pools > false > > {code} > a related question is here: > http://stackoverflow.com/questions/26488564/hadoop-yarn-why-the-queue-cannot-be-deleted-after-i-revise-my-fair-scheduler-xm -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[ https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951560#comment-14951560 ] forrestchen commented on YARN-4022: --- Could anyone help review this issue please? > queue not remove from webpage(/cluster/scheduler) when delete queue in > xxx-scheduler.xml > > > Key: YARN-4022 > URL: https://issues.apache.org/jira/browse/YARN-4022 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.7.1 >Reporter: forrestchen > Labels: scheduler > Attachments: YARN-4022.001.patch, YARN-4022.002.patch, > YARN-4022.003.patch, YARN-4022.004.patch > > > When I delete an existing queue by modify the xxx-schedule.xml, I can still > see the queue information block in webpage(/cluster/scheduler) though the > 'Min Resources' items all become to zero and have no item of 'Max Running > Applications'. > I can still submit an application to the deleted queue and the application > will run using 'root.default' queue instead, but submit to an un-exist queue > will cause an exception. > My expectation is the deleted queue will not displayed in webpage and submit > application to the deleted queue will act just like the queue doesn't exist. > PS: There's no application running in the queue I delete. > Some related config in yarn-site.xml: > {code} > > yarn.scheduler.fair.user-as-default-queue > false > > > yarn.scheduler.fair.allow-undeclared-pools > false > > {code} > a related question is here: > http://stackoverflow.com/questions/26488564/hadoop-yarn-why-the-queue-cannot-be-deleted-after-i-revise-my-fair-scheduler-xm -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[ https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736390#comment-14736390 ] Hadoop QA commented on YARN-4022: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 16m 55s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | | {color:green}+1{color} | javac | 8m 4s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 15s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 0m 55s | The applied patch generated 1 new checkstyle issues (total was 85, now 84). | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 32s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 32s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:red}-1{color} | yarn tests | 54m 21s | Tests failed in hadoop-yarn-server-resourcemanager. | | | | 94m 33s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA | | | hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12754830/YARN-4022.003.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 94cf7ab | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/9053/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/9053/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/9053/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/9053/console | This message was automatically generated. > queue not remove from webpage(/cluster/scheduler) when delete queue in > xxx-scheduler.xml > > > Key: YARN-4022 > URL: https://issues.apache.org/jira/browse/YARN-4022 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.7.1 >Reporter: forrestchen > Labels: scheduler > Attachments: YARN-4022.001.patch, YARN-4022.002.patch, > YARN-4022.003.patch > > > When I delete an existing queue by modify the xxx-schedule.xml, I can still > see the queue information block in webpage(/cluster/scheduler) though the > 'Min Resources' items all become to zero and have no item of 'Max Running > Applications'. > I can still submit an application to the deleted queue and the application > will run using 'root.default' queue instead, but submit to an un-exist queue > will cause an exception. > My expectation is the deleted queue will not displayed in webpage and submit > application to the deleted queue will act just like the queue doesn't exist. > PS: There's no application running in the queue I delete. > Some related config in yarn-site.xml: > {code} > > yarn.scheduler.fair.user-as-default-queue > false > > > yarn.scheduler.fair.allow-undeclared-pools > false > > {code} > a related question is here: > http://stackoverflow.com/questions/26488564/hadoop-yarn-why-the-queue-cannot-be-deleted-after-i-revise-my-fair-scheduler-xm -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[ https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736590#comment-14736590 ] Hadoop QA commented on YARN-4022: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 17m 4s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | | {color:green}+1{color} | javac | 7m 55s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 54s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 22s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | checkstyle | 0m 51s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 26s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 28s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | yarn tests | 54m 17s | Tests passed in hadoop-yarn-server-resourcemanager. | | | | 93m 54s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12754857/YARN-4022.004.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 0113e45 | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/9054/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/9054/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/9054/console | This message was automatically generated. > queue not remove from webpage(/cluster/scheduler) when delete queue in > xxx-scheduler.xml > > > Key: YARN-4022 > URL: https://issues.apache.org/jira/browse/YARN-4022 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.7.1 >Reporter: forrestchen > Labels: scheduler > Attachments: YARN-4022.001.patch, YARN-4022.002.patch, > YARN-4022.003.patch, YARN-4022.004.patch > > > When I delete an existing queue by modify the xxx-schedule.xml, I can still > see the queue information block in webpage(/cluster/scheduler) though the > 'Min Resources' items all become to zero and have no item of 'Max Running > Applications'. > I can still submit an application to the deleted queue and the application > will run using 'root.default' queue instead, but submit to an un-exist queue > will cause an exception. > My expectation is the deleted queue will not displayed in webpage and submit > application to the deleted queue will act just like the queue doesn't exist. > PS: There's no application running in the queue I delete. > Some related config in yarn-site.xml: > {code} > > yarn.scheduler.fair.user-as-default-queue > false > > > yarn.scheduler.fair.allow-undeclared-pools > false > > {code} > a related question is here: > http://stackoverflow.com/questions/26488564/hadoop-yarn-why-the-queue-cannot-be-deleted-after-i-revise-my-fair-scheduler-xm -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[ https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14734751#comment-14734751 ] Hadoop QA commented on YARN-4022: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 16m 20s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | | {color:green}+1{color} | javac | 7m 38s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 59s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 0m 50s | The applied patch generated 11 new checkstyle issues (total was 85, now 94). | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 29s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 33s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:red}-1{color} | yarn tests | 54m 7s | Tests failed in hadoop-yarn-server-resourcemanager. | | | | 92m 55s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12754618/YARN-4022.002.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 435f935 | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/9035/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/9035/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/9035/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/9035/console | This message was automatically generated. > queue not remove from webpage(/cluster/scheduler) when delete queue in > xxx-scheduler.xml > > > Key: YARN-4022 > URL: https://issues.apache.org/jira/browse/YARN-4022 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.7.1 >Reporter: forrestchen > Labels: scheduler > Attachments: YARN-4022.001.patch, YARN-4022.002.patch > > > When I delete an existing queue by modify the xxx-schedule.xml, I can still > see the queue information block in webpage(/cluster/scheduler) though the > 'Min Resources' items all become to zero and have no item of 'Max Running > Applications'. > I can still submit an application to the deleted queue and the application > will run using 'root.default' queue instead, but submit to an un-exist queue > will cause an exception. > My expectation is the deleted queue will not displayed in webpage and submit > application to the deleted queue will act just like the queue doesn't exist. > PS: There's no application running in the queue I delete. > Some related config in yarn-site.xml: > {code} > > yarn.scheduler.fair.user-as-default-queue > false > > > yarn.scheduler.fair.allow-undeclared-pools > false > > {code} > a related question is here: > http://stackoverflow.com/questions/26488564/hadoop-yarn-why-the-queue-cannot-be-deleted-after-i-revise-my-fair-scheduler-xm -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[ https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14734580#comment-14734580 ] Hadoop QA commented on YARN-4022: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 15m 28s | Findbugs (version ) appears to be broken on trunk. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | | {color:green}+1{color} | javac | 7m 39s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 3s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 24s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | checkstyle | 0m 25s | There were no new checkstyle issues. | | {color:red}-1{color} | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 32s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 30s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:red}-1{color} | yarn tests | 52m 30s | Tests failed in hadoop-yarn-server-resourcemanager. | | | | 90m 8s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler | | | hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService | | | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification | | | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs | | | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerFairShare | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12754601/YARN-4022.001.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 435f935 | | whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/9032/artifact/patchprocess/whitespace.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/9032/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/9032/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/9032/console | This message was automatically generated. > queue not remove from webpage(/cluster/scheduler) when delete queue in > xxx-scheduler.xml > > > Key: YARN-4022 > URL: https://issues.apache.org/jira/browse/YARN-4022 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: forrestchen > Labels: scheduler > Attachments: YARN-4022.001.patch > > > When I delete an existing queue by modify the xxx-schedule.xml, I can still > see the queue information block in webpage(/cluster/scheduler) though the > 'Min Resources' items all become to zero and have no item of 'Max Running > Applications'. > I can still submit an application to the deleted queue and the application > will run using 'root.default' queue instead, but submit to an un-exist queue > will cause an exception. > My expectation is the deleted queue will not displayed in webpage and submit > application to the deleted queue will act just like the queue doesn't exist. > PS: There's no application running in the queue I delete. > Some related config in yarn-site.xml: > {code} > > yarn.scheduler.fair.user-as-default-queue > false > > > yarn.scheduler.fair.allow-undeclared-pools > false > > {code} > a related question is here: > http://stackoverflow.com/questions/26488564/hadoop-yarn-why-the-queue-cannot-be-deleted-after-i-revise-my-fair-scheduler-xm -- This message was sent by Atlassian JIRA (v6.3.4#6332)