[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681998#comment-14681998 ] Hudson commented on YARN-3887: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #273 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/273/]) YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Fix For: 2.8.0 > > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch, > 0006-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681964#comment-14681964 ] Hudson commented on YARN-3887: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2211 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2211/]) YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Fix For: 2.8.0 > > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch, > 0006-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681917#comment-14681917 ] Hudson commented on YARN-3887: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2230 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2230/]) YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Fix For: 2.8.0 > > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch, > 0006-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681920#comment-14681920 ] Hudson commented on YARN-3887: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #281 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/281/]) YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Fix For: 2.8.0 > > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch, > 0006-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681699#comment-14681699 ] Sunil G commented on YARN-3887: --- Thank you very much [~jianhe] for reviewing and committing the same. [~leftnoteasy] and [~rohithsharma], thank you for the reviews. > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Fix For: 2.8.0 > > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch, > 0006-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681650#comment-14681650 ] Hudson commented on YARN-3887: -- FAILURE: Integrated in Hadoop-Yarn-trunk #1014 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1014/]) YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Fix For: 2.8.0 > > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch, > 0006-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681645#comment-14681645 ] Hudson commented on YARN-3887: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #284 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/284/]) YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Fix For: 2.8.0 > > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch, > 0006-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681271#comment-14681271 ] Hudson commented on YARN-3887: -- FAILURE: Integrated in Hadoop-trunk-Commit #8282 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8282/]) YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Fix For: 2.8.0 > > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch, > 0006-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681059#comment-14681059 ] Rohith Sharma K S commented on YARN-3887: - +1 lgtm > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch, > 0006-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680935#comment-14680935 ] Jian He commented on YARN-3887: --- +1, [~rohithsharma], do you have any other comments > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch, > 0006-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680137#comment-14680137 ] Sunil G commented on YARN-3887: --- Test failures are unrelated to this patch. > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch, > 0006-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14679087#comment-14679087 ] Hadoop QA commented on YARN-3887: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 16m 10s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 43s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 42s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 0m 49s | The applied patch generated 2 new checkstyle issues (total was 266, now 268). | | {color:red}-1{color} | whitespace | 0m 7s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 21s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 28s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:red}-1{color} | yarn tests | 53m 19s | Tests failed in hadoop-yarn-server-resourcemanager. | | | | 91m 38s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation | | | hadoop.yarn.server.resourcemanager.TestRMAdminService | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12749454/0006-YARN-3887.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 8f73bdd | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/8804/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt | | whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/8804/artifact/patchprocess/whitespace.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/8804/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/8804/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8804/console | This message was automatically generated. > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch, > 0006-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653002#comment-14653002 ] Rohith Sharma K S commented on YARN-3887: - One comment # TreeSet will throw NullPointerException while adding/removing null object. Suppose, SchedulintApplicationAttempt is not created then {{application.getCurrentAppAttempt()}} will be null which would throw NPE. I think this has to be handled in {{AbstractComparatorOrderingPolicy#removeSchedulableEntity}} and {{AbstractComparatorOrderingPolicy#addSchedulableEntity}} > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14652874#comment-14652874 ] Jian He commented on YARN-3887: --- thanks Sunil, for below, I think we can add a appPriorityUpdated method in OrderingPolicy and follow the implementation of demandUpdated method. Doing that, the synchronization of queue may not be needed. {code} synchronized (queue) { queue.getOrderingPolicy().removeSchedulableEntity( application.getCurrentAppAttempt()); // Update new priority in SchedulerApplication application.setPriority(appPriority); queue.getOrderingPolicy().addSchedulableEntity( application.getCurrentAppAttempt()); }{code} > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14650659#comment-14650659 ] Sunil G commented on YARN-3887: --- This test case is passing locally. Not related to this patch. > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14649107#comment-14649107 ] Hadoop QA commented on YARN-3887: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 16m 3s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 39s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 37s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 22s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 0m 46s | The applied patch generated 2 new checkstyle issues (total was 263, now 265). | | {color:green}+1{color} | whitespace | 0m 7s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 21s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 27s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:red}-1{color} | yarn tests | 52m 48s | Tests failed in hadoop-yarn-server-resourcemanager. | | | | 90m 45s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12748148/0005-YARN-3887.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 93d50b7 | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/8736/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/8736/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/8736/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8736/console | This message was automatically generated. > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14649045#comment-14649045 ] Hadoop QA commented on YARN-3887: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 16m 43s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 46s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 46s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 22s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 0m 48s | The applied patch generated 2 new checkstyle issues (total was 263, now 265). | | {color:green}+1{color} | whitespace | 0m 5s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 21s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 26s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | yarn tests | 53m 2s | Tests passed in hadoop-yarn-server-resourcemanager. | | | | 91m 55s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12748137/0004-YARN-3887.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 93d50b7 | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/8734/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/8734/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/8734/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8734/console | This message was automatically generated. > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch, 0005-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14648969#comment-14648969 ] Rohith Sharma K S commented on YARN-3887: - [~sunilg] thanks for updating patch One comment # The below code should not be synchronized. If we have synchronized, then there is very high chance of deadlock. The locking order should be always from {{stateMachine --> RMStateStore}} but below code locks in {{RMStateStore --> stateMachine --> RMStateStore}} which causes deadlock. For more discussion refer YARN-2946 {code} + public synchronized void updateApplicationStateSynchronously( + ApplicationStateData appState) { +handleStoreEvent(new RMStateUpdateAppEvent(appState)); + } {code} > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch, 0004-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14648089#comment-14648089 ] Hadoop QA commented on YARN-3887: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 19m 58s | Findbugs (version ) appears to be broken on trunk. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 10m 35s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 11m 25s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 31s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | checkstyle | 0m 36s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 7s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 43s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 37s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 38s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:red}-1{color} | yarn tests | 53m 4s | Tests failed in hadoop-yarn-server-resourcemanager. | | | | 100m 19s | | \\ \\ || Reason || Tests || | Timed out tests | org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12748024/0003-YARN-3887.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 8acb30b | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/8717/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/8717/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8717/console | This message was automatically generated. > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14647984#comment-14647984 ] Wangda Tan commented on YARN-3887: -- [~sunilg], I took a quick look at the patch, some comments: - I think you should acquire synchronized lock of LeafQueue when reordering schedulableEntity. - SchedulerApplication / SchedulerApplicationAttempt.priority should be volatile, please let me know if you think it's not necessary. - Have you considered the case when SchedulerApplicationAttempt is not created? If you allow edit SchedulerApplication.priority alone (When SchedulerAttempt is not created), you may need to consider one race condition is: SchedulerApplication changing priority and creating SchedulerAttempt happen together, which may need to acquire CapacityScheduler lock. > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch, > 0003-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14646676#comment-14646676 ] Jian He commented on YARN-3887: --- sounds good to me > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14645566#comment-14645566 ] Rohith Sharma K S commented on YARN-3887: - Your understanding is correct. I was meant to say to have new synchronous API like {{updateApplicationStateSynchronizly}} in RMStateStore. [~jianhe] what do you think having new synchronous api in RMstatstore? > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14645448#comment-14645448 ] Sunil G commented on YARN-3887: --- Thank you [~jianhe] for the comments. I will update a patch addressing the same. [~rohithsharma] If I understood your comment correctly, we may get a scenario when RMStateStore is slow in processing events and RM restart happened. Hence priority will not be updated in state store. {code} public synchronized void updateApplicationState( ApplicationStateData appState) { dispatcher.getEventHandler().handle(new RMStateUpdateAppEvent(appState)); } {code} This api is using the event handler to process the new RMStateUpdate. As mentioned by you, If we see {{storeRMDelegationToken}}, it calls {{handleStoreEvent}} directly. So to summarize, could we have a new api in RMStateStore as follows {code} public synchronized void updateApplicationStateSynchronizly( ApplicationStateData appState) { handleStoreEvent(new RMStateUpdateAppEvent(appState)); } {code} and use it from scheduler to avoid this case. Thoughts? > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14645425#comment-14645425 ] Rohith Sharma K S commented on YARN-3887: - thanks [~sunilg] for updating patch. Some comments # The invocation {{rmContext.getStateStore().updateApplicationState(appState);}} is asynchronous. So I feel stil there will be corner case would ocure where priority has set in scheduler but not updated to RMStateSstore. So if any RM switch/Restart would end up in resulting in old priority set. I think this particular invocation should be should be synchronous like any others API's E.g: {{storeRMDelegationToken}}, {{storeRMDTMasterKey}} > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14645172#comment-14645172 ] Hadoop QA commented on YARN-3887: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 18m 0s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 44s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 38s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 24s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 0m 49s | The applied patch generated 1 new checkstyle issues (total was 181, now 182). | | {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 23s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 25s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | yarn tests | 52m 41s | Tests passed in hadoop-yarn-server-resourcemanager. | | | | 92m 43s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12747616/0002-YARN-3887.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 69b0957 | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/8698/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/8698/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/8698/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8698/console | This message was automatically generated. > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14645062#comment-14645062 ] Jian He commented on YARN-3887: --- thanks for updating, looks good overall, few comments: - SchedulerApplicationAttempt#appPriority can also be volatile ? - testRMRestartWithChangeInPriority- Since the state store update is asynchronous, the test may fail if the state is not updated immediately, we may wait until the state is updated? > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch, 0002-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14644008#comment-14644008 ] Rohith Sharma K S commented on YARN-3887: - Hi [~sunilg] For REST support, proto changes are not done, but for admin/user proto changes to be done. So I mean it can be done separate jira > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643945#comment-14643945 ] Sunil G commented on YARN-3887: --- Hi [~jianhe] and [~rohithsharma] We also raised YARN-3970 which is planned use same scheduler apis to get/set priority of an application at runtime from REST. So if we can do a common server side implementation in this ticket, it can be used in others. Please share thoughts. Meantime I will update a patch addressing other comments. > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643931#comment-14643931 ] Rohith Sharma K S commented on YARN-3887: - Hi Jian He, bq. Do you plan to do client side changes as part of this jira ? YARN-3250 is planning to do changes for admin and user CLI i.e ApplicationClientProtocol. This jira is intended for only scheduler side chagnes support for API's. In YARN-3250, will be using these exposed API's and implementing it. Current plan, Admin/User both have previlages to change priority of applications. More API's from Admin and User to be discussed in yarn-3250. > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643701#comment-14643701 ] Jian He commented on YARN-3887: --- thanks Sunil ! some comments on the patch: - Do you plan to do client side changes as part of this jira ? - RMAppUpdatePriorityEvent - RMApp may receive this event at many other states other than RUNNING state. In that case, the state-machine will throw InvalidEventException. I think we do not need to send event to RMApp, all it does is just get the application submission context and set the priority. This can be done at clientRMService. Similarly, the event to state-store can be sent directly from clientRMService. - CapacityScheuduler#updateApplicationPriority does not need to be synchronized ? > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3887) Support for changing Application priority during runtime
[ https://issues.apache.org/jira/browse/YARN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637201#comment-14637201 ] Hadoop QA commented on YARN-3887: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 16m 58s | Findbugs (version ) appears to be broken on trunk. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 49s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 57s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 24s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | checkstyle | 0m 22s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 2s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 22s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 25s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:red}-1{color} | yarn tests | 51m 0s | Tests failed in hadoop-yarn-server-resourcemanager. | | | | 90m 1s | | \\ \\ || Reason || Tests || | Timed out tests | org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup | | | org.apache.hadoop.yarn.server.resourcemanager.TestRM | | | org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12746548/0001-YARN-3887.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 4025326 | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/8612/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/8612/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8612/console | This message was automatically generated. > Support for changing Application priority during runtime > > > Key: YARN-3887 > URL: https://issues.apache.org/jira/browse/YARN-3887 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-3887.patch > > > After YARN-2003, adding support to change priority of an application after > submission. This ticket will handle the server side implementation for same. > A new RMAppEvent will be created to handle this, and will be common for all > schedulers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)