[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906297#comment-14906297 ] Hudson commented on YARN-4044: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2351 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2351/]) YARN-4044. Running applications information changes such as movequeue is not published to TimeLine server. (Sunil G via rohithsharmaks) (rohithsharmaks: rev a9aafad12b1d2f67e55e09a6fa261d61789c9d7e) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/ApplicationUpdatedEvent.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TestSystemMetricsPublisher.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/metrics/ApplicationMetricsConstants.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryManagerOnTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsPublisher.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsEventType.java * hadoop-yarn-project/CHANGES.txt > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Fix For: 2.8.0 > > Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch, > 0003-YARN-4044.patch, 0004-YARN-4044.patch, 0005-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906193#comment-14906193 ] Hudson commented on YARN-4044: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2378 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2378/]) YARN-4044. Running applications information changes such as movequeue is not published to TimeLine server. (Sunil G via rohithsharmaks) (rohithsharmaks: rev a9aafad12b1d2f67e55e09a6fa261d61789c9d7e) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsPublisher.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/ApplicationUpdatedEvent.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsEventType.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryManagerOnTimelineStore.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/metrics/ApplicationMetricsConstants.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TestSystemMetricsPublisher.java > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Fix For: 2.8.0 > > Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch, > 0003-YARN-4044.patch, 0004-YARN-4044.patch, 0005-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906086#comment-14906086 ] Hudson commented on YARN-4044: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #412 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/412/]) YARN-4044. Running applications information changes such as movequeue is not published to TimeLine server. (Sunil G via rohithsharmaks) (rohithsharmaks: rev a9aafad12b1d2f67e55e09a6fa261d61789c9d7e) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryManagerOnTimelineStore.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/metrics/ApplicationMetricsConstants.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TestSystemMetricsPublisher.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsEventType.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/ApplicationUpdatedEvent.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsPublisher.java > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Fix For: 2.8.0 > > Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch, > 0003-YARN-4044.patch, 0004-YARN-4044.patch, 0005-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906041#comment-14906041 ] Hudson commented on YARN-4044: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #440 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/440/]) YARN-4044. Running applications information changes such as movequeue is not published to TimeLine server. (Sunil G via rohithsharmaks) (rohithsharmaks: rev a9aafad12b1d2f67e55e09a6fa261d61789c9d7e) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TestSystemMetricsPublisher.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/ApplicationUpdatedEvent.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryManagerOnTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsEventType.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsPublisher.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/metrics/ApplicationMetricsConstants.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Fix For: 2.8.0 > > Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch, > 0003-YARN-4044.patch, 0004-YARN-4044.patch, 0005-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906028#comment-14906028 ] Hudson commented on YARN-4044: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #1172 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1172/]) YARN-4044. Running applications information changes such as movequeue is not published to TimeLine server. (Sunil G via rohithsharmaks) (rohithsharmaks: rev a9aafad12b1d2f67e55e09a6fa261d61789c9d7e) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TestSystemMetricsPublisher.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryManagerOnTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsPublisher.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/ApplicationUpdatedEvent.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsEventType.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/metrics/ApplicationMetricsConstants.java > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Fix For: 2.8.0 > > Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch, > 0003-YARN-4044.patch, 0004-YARN-4044.patch, 0005-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14905947#comment-14905947 ] Hudson commented on YARN-4044: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #432 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/432/]) YARN-4044. Running applications information changes such as movequeue is not published to TimeLine server. (Sunil G via rohithsharmaks) (rohithsharmaks: rev a9aafad12b1d2f67e55e09a6fa261d61789c9d7e) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/ApplicationUpdatedEvent.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsPublisher.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsEventType.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/metrics/ApplicationMetricsConstants.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryManagerOnTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TestSystemMetricsPublisher.java > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Fix For: 2.8.0 > > Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch, > 0003-YARN-4044.patch, 0004-YARN-4044.patch, 0005-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14905913#comment-14905913 ] Hudson commented on YARN-4044: -- FAILURE: Integrated in Hadoop-trunk-Commit #8509 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8509/]) YARN-4044. Running applications information changes such as movequeue is not published to TimeLine server. (Sunil G via rohithsharmaks) (rohithsharmaks: rev a9aafad12b1d2f67e55e09a6fa261d61789c9d7e) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryManagerOnTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/ApplicationUpdatedEvent.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsPublisher.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/metrics/ApplicationMetricsConstants.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TestSystemMetricsPublisher.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsEventType.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Fix For: 2.8.0 > > Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch, > 0003-YARN-4044.patch, 0004-YARN-4044.patch, 0005-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14905897#comment-14905897 ] Rohith Sharma K S commented on YARN-4044: - +1 for the latest patch.. verified the changes in deploying in cluster too... committing shortly.. > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch, > 0003-YARN-4044.patch, 0004-YARN-4044.patch, 0005-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[
https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14852731#comment-14852731
]
Hadoop QA commented on YARN-4044:
-
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 21m 27s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 8m 59s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 11m 27s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 48s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 2s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 45s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 40s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 56s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 4m 5s | Tests passed in
hadoop-yarn-server-applicationhistoryservice. |
| {color:green}+1{color} | yarn tests | 0m 27s | Tests passed in
hadoop-yarn-server-common. |
| {color:green}+1{color} | yarn tests | 60m 29s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 115m 35s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12758372/0005-YARN-4044.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / a7201d6 |
| hadoop-yarn-server-applicationhistoryservice test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9207/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
|
| hadoop-yarn-server-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9207/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9207/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9207/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9207/console |
This message was automatically generated.
> Running applications information changes such as movequeue is not published
> to TimeLine server
> --
>
> Key: YARN-4044
> URL: https://issues.apache.org/jira/browse/YARN-4044
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, timelineserver
>Affects Versions: 2.7.0
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Critical
> Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch,
> 0003-YARN-4044.patch, 0004-YARN-4044.patch, 0005-YARN-4044.patch
>
>
> SystemMetricsPublisher need to expose an appUpdated api to update any change
> for a running application.
> Events can be
> - change of queue for a running application.
> - change of application priority for a running application.
> This ticket intends to handle both RM and timeline side changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[
https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14805267#comment-14805267
]
Naganarasimha G R commented on YARN-4044:
-
Thanks for pointing out [~sunilg], Now i recollect why its like that, its
because of {{CompositEventHandler in MultiThreadedDispatcher}} which is used to
dispatch the events in multiple threads and all the events related to a given
app they want to send it in one thread. And equals we cannot ensure using appid
hence equals is not overwritten. I feel the design is not appropriate here but
anyway i am modifying the design so modifications in your current patch should
for it should be ok.
> Running applications information changes such as movequeue is not published
> to TimeLine server
> --
>
> Key: YARN-4044
> URL: https://issues.apache.org/jira/browse/YARN-4044
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, timelineserver
>Affects Versions: 2.7.0
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Critical
> Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch,
> 0003-YARN-4044.patch, 0004-YARN-4044.patch
>
>
> SystemMetricsPublisher need to expose an appUpdated api to update any change
> for a running application.
> Events can be
> - change of queue for a running application.
> - change of application priority for a running application.
> This ticket intends to handle both RM and timeline side changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[
https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14805004#comment-14805004
]
Sunil G commented on YARN-4044:
---
Hi [~Naganarasimha Garla]
Yes. we can add more events in test case to verify the order, this can be done.
I will update.
As I see point 2, all {{SystemMetricsEvent}} are overriding {{hashCode}} along
with the new {{ApplicationUpdatedEvent}}. And if you see
{{AppAttemptRegisteredEvent}}, there also the hashCode is overridden with
*appId*, not *appAttemptId*.
{code}
@Override
public int hashCode() {
return appAttemptId.getApplicationId().hashCode();
}
{code}
Also sharing code snippet from {{ContainerCreatedEvent}}
{code}
@Override
public int hashCode() {
return containerId.getApplicationAttemptId().getApplicationId().hashCode();
}
{code}
So the intention looks like is to treat all {{SystemMetricsEvent}} similar I
feel. [~rohithsharma] thoughts?
> Running applications information changes such as movequeue is not published
> to TimeLine server
> --
>
> Key: YARN-4044
> URL: https://issues.apache.org/jira/browse/YARN-4044
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, timelineserver
>Affects Versions: 2.7.0
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Critical
> Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch,
> 0003-YARN-4044.patch, 0004-YARN-4044.patch
>
>
> SystemMetricsPublisher need to expose an appUpdated api to update any change
> for a running application.
> Events can be
> - change of queue for a running application.
> - change of application priority for a running application.
> This ticket intends to handle both RM and timeline side changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[
https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14804943#comment-14804943
]
Naganarasimha G R commented on YARN-4044:
-
Hi [~sunilg],
Thanks for updating the patch, few nits
# As discussed earlier, we are depending on one of the feature of HistoryServer
to return the latest event. So it would be better to cover in our test case as,
if there is break in that feature our testcase can catch it. i.e to add
additional events modifying the queue / priority (may be one modification for
queue and other for Priority).
# ApplicationUpdatedEvent has overridden {{hashCode}} but not {{equals}}. In
the first place do we require to over ride the hashcode method ?
> Running applications information changes such as movequeue is not published
> to TimeLine server
> --
>
> Key: YARN-4044
> URL: https://issues.apache.org/jira/browse/YARN-4044
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, timelineserver
>Affects Versions: 2.7.0
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Critical
> Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch,
> 0003-YARN-4044.patch, 0004-YARN-4044.patch
>
>
> SystemMetricsPublisher need to expose an appUpdated api to update any change
> for a running application.
> Events can be
> - change of queue for a running application.
> - change of application priority for a running application.
> This ticket intends to handle both RM and timeline side changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[
https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14804568#comment-14804568
]
Hadoop QA commented on YARN-4044:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 18m 29s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 7m 52s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 15s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 35s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 32s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 36s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 3m 54s | Tests passed in
hadoop-yarn-server-applicationhistoryservice. |
| {color:green}+1{color} | yarn tests | 0m 26s | Tests passed in
hadoop-yarn-server-common. |
| {color:red}-1{color} | yarn tests | 53m 49s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 102m 27s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.yarn.server.resourcemanager.webapp.TestAppPage |
| |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification |
| | hadoop.yarn.server.resourcemanager.TestRMAdminService |
| Timed out tests |
org.apache.hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
|
| |
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
|
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12757135/0004-YARN-4044.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 3f82f58 |
| hadoop-yarn-server-applicationhistoryservice test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9197/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
|
| hadoop-yarn-server-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9197/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9197/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9197/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9197/console |
This message was automatically generated.
> Running applications information changes such as movequeue is not published
> to TimeLine server
> --
>
> Key: YARN-4044
> URL: https://issues.apache.org/jira/browse/YARN-4044
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, timelineserver
>Affects Versions: 2.7.0
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Critical
> Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch,
> 0003-YARN-4044.patch, 0004-YARN-4044.patch
>
>
> SystemMetricsPublisher need to expose an appUpdated api to update any change
> for a running application.
> Events can be
> - change of queue for a running application.
> - change of application priority for a running application.
> This ticket intends to handle both RM and timeline side changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[
https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746789#comment-14746789
]
Naganarasimha G R commented on YARN-4044:
-
Hi [~sunilg],
I think overall the patch is fine just for few nits :
# In {{ApplicationHistoryManagerOnTimelineStore}} if check is not required as
in any case we filling both Priority and QueueName when there is appupdate
event.
# We need to update the test case for
{{ApplicationHistoryManagerOnTimelineStore}} modifications
> Running applications information changes such as movequeue is not published
> to TimeLine server
> --
>
> Key: YARN-4044
> URL: https://issues.apache.org/jira/browse/YARN-4044
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, timelineserver
>Affects Versions: 2.7.0
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Critical
> Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch,
> 0003-YARN-4044.patch
>
>
> SystemMetricsPublisher need to expose an appUpdated api to update any change
> for a running application.
> Events can be
> - change of queue for a running application.
> - change of application priority for a running application.
> This ticket intends to handle both RM and timeline side changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[
https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746083#comment-14746083
]
Hadoop QA commented on YARN-4044:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 54s | Findbugs (version ) appears to
be broken on trunk. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 53s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 59s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 3s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 32s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | yarn tests | 3m 26s | Tests failed in
hadoop-yarn-server-applicationhistoryservice. |
| {color:green}+1{color} | yarn tests | 0m 25s | Tests passed in
hadoop-yarn-server-common. |
| {color:red}-1{color} | yarn tests | 51m 1s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 96m 51s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.timeline.webapp.TestTimelineWebServices |
| | hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher |
| | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue |
| |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
|
| Timed out tests |
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
|
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12756041/0003-YARN-4044.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / ae5308f |
| hadoop-yarn-server-applicationhistoryservice test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9147/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
|
| hadoop-yarn-server-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9147/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9147/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9147/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9147/console |
This message was automatically generated.
> Running applications information changes such as movequeue is not published
> to TimeLine server
> --
>
> Key: YARN-4044
> URL: https://issues.apache.org/jira/browse/YARN-4044
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, timelineserver
>Affects Versions: 2.7.0
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Critical
> Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch,
> 0003-YARN-4044.patch
>
>
> SystemMetricsPublisher need to expose an appUpdated api to update any change
> for a running application.
> Events can be
> - change of queue for a running application.
> - change of application priority for a running application.
> This ticket intends to handle both RM and timeline side changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737639#comment-14737639 ] Naganarasimha G R commented on YARN-4044: - Thanks for the update [~sunilg], My thoughts initially was also same but may be I could have conveyed better, i meant it would good for analysis if atleast if application updates are captured in ATS and my intention was not to show in UI. > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737303#comment-14737303 ] Sunil G commented on YARN-4044: --- Thank you [~Naganarasimha Garla] for the input. Idea of publish each app changed event (priority/queue) is good. But current V1 GUI doesnt have support to display all changed entities of application in UI. I discussed with [~jianhe] also offline. As per our discussion, we need not need to concentrate on showing this changed entities in UI now. It will make current UI complex. In future UI work, we can handle this in a better human readable form (application timelilne graph maybe). However we can have changed events in YARN level rather than entity overwrite, and we can show last changed event for now. I will update a patch based on this understanding. > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729444#comment-14729444 ] Naganarasimha G R commented on YARN-4044: - [~sunilg], As per the offline discussion we had, currently you are populating the entityInfo Object with the modified priority or the queue because of which ATS is able to provide the updated info. So yes both of my queries gets answered with it . But one point which i can think of here is would it be better to capture the modified priority and queue (if any) as part of eventInfo ? so that initial submission information as well as modification information will be captured which might be helpful later in analysis. > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[
https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728938#comment-14728938
]
Sunil G commented on YARN-4044:
---
Thank You [~Naganarasimha] for sharing the comments.
bq.ApplicationHistoryManagerOnTimelineStore.convertToApplicationReport needs to
capture the event ApplicationMetricsConstants.UPDATED_EVENT_TYPE
I feel this is not needed. We are already not handling
{{ACLS_UPDATED_EVENT_TYPE}} also. I think we use different handling in
{{convertToApplicationReport}} for CREATED_EVENT_TYPE and FINISHED_EVENT_TYPE
to record appStarted time and appFinished time (plus few other finished details)
I used {{entityInfo}} to store Queue and Priority details. Hence the first part
of code in {{convertToApplicationReport}} covers reading that details and will
fed to {{ApplicationReportExt}} object. Please share your opinion.
bq.we need to check return the report with the information present in latest
ApplicationMetricsConstants.UPDATED_EVENT_TYPE
One doubt here. We already are using {{EnumSet.allOf(Field.class)}} in
{{ApplicationHistoryManagerOnTimelineStore.getApplication}} and hence it has
{{LAST_EVENT_ONLY}} field. So we will read the last saved entity. Please help
to correct me if I am wrong.
> Running applications information changes such as movequeue is not published
> to TimeLine server
> --
>
> Key: YARN-4044
> URL: https://issues.apache.org/jira/browse/YARN-4044
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, timelineserver
>Affects Versions: 2.7.0
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Critical
> Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch
>
>
> SystemMetricsPublisher need to expose an appUpdated api to update any change
> for a running application.
> Events can be
> - change of queue for a running application.
> - change of application priority for a running application.
> This ticket intends to handle both RM and timeline side changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[
https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728639#comment-14728639
]
Naganarasimha G R commented on YARN-4044:
-
Hi [~sunilg], IIUC intention of this jira is to show in the ATS UI with the
modifed queue or priority if any right ?. If so it requires modification in
{{ApplicationHistoryManagerOnTimelineStore.convertToApplicationReport}} needs
to capture the event {{ApplicationMetricsConstants.UPDATED_EVENT_TYPE}} and
update the ApplicationReport object. At the same time we need to ensure that if
multiple application updates are done then we need to show the latest update in
the UI. As per the logic what i see in
{{LeveldbTimelineStore.getEntity(String, String, Long, EnumSet,
LeveldbIterator, byte[], int)}} seems like all the events are captured and
returned in which case we need to check return the report with the information
present in latest {{ApplicationMetricsConstants.UPDATED_EVENT_TYPE}}
> Running applications information changes such as movequeue is not published
> to TimeLine server
> --
>
> Key: YARN-4044
> URL: https://issues.apache.org/jira/browse/YARN-4044
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, timelineserver
>Affects Versions: 2.7.0
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Critical
> Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch
>
>
> SystemMetricsPublisher need to expose an appUpdated api to update any change
> for a running application.
> Events can be
> - change of queue for a running application.
> - change of application priority for a running application.
> This ticket intends to handle both RM and timeline side changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727837#comment-14727837 ] Sunil G commented on YARN-4044: --- Hi [~rohithsharma] Could you please take a look on this patch. > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14711147#comment-14711147 ] Sunil G commented on YARN-4044: --- Test case failures are not related. > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[
https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14710984#comment-14710984
]
Hadoop QA commented on YARN-4044:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 49s | Findbugs (version ) appears to
be broken on trunk. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 57s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 4s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 59s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 32s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 36s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 33s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 0m 25s | Tests passed in
hadoop-yarn-server-common. |
| {color:red}-1{color} | yarn tests | 50m 39s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 92m 0s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart |
| | hadoop.yarn.server.resourcemanager.TestRM |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12752166/0002-YARN-4044.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / af78767 |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/8903/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-server-common.html
|
| hadoop-yarn-server-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8903/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8903/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8903/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8903/console |
This message was automatically generated.
> Running applications information changes such as movequeue is not published
> to TimeLine server
> --
>
> Key: YARN-4044
> URL: https://issues.apache.org/jira/browse/YARN-4044
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, timelineserver
>Affects Versions: 2.7.0
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Critical
> Attachments: 0001-YARN-4044.patch, 0002-YARN-4044.patch
>
>
> SystemMetricsPublisher need to expose an appUpdated api to update any change
> for a running application.
> Events can be
> - change of queue for a running application.
> - change of application priority for a running application.
> This ticket intends to handle both RM and timeline side changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706234#comment-14706234 ] Sunil G commented on YARN-4044: --- Thank you [~rohithsharma] Yes. I have verified this in real cluster. I will upload few screen shots later. > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Attachments: 0001-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706224#comment-14706224 ] Rohith Sharma K S commented on YARN-4044: - Thanks [~sunilg] for the patch.. The patch mostly looks good to me.. Have you verified in the real cluster? > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Attachments: 0001-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14703424#comment-14703424 ] Sunil G commented on YARN-4044: --- Test case failures are unrelated to this patch. > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Attachments: 0001-YARN-4044.patch > > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[
https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14695646#comment-14695646
]
Hadoop QA commented on YARN-4044:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 17m 30s | Pre-patch trunk has 7 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 55s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 52s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 25s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 11s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 23s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 33s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 0m 24s | Tests passed in
hadoop-yarn-server-common. |
| {color:red}-1{color} | yarn tests | 53m 30s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 95m 20s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12750296/0001-YARN-4044.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 53bef9c |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/8843/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-server-common.html
|
| hadoop-yarn-server-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8843/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8843/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8843/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8843/console |
This message was automatically generated.
> Running applications information changes such as movequeue is not published
> to TimeLine server
> --
>
> Key: YARN-4044
> URL: https://issues.apache.org/jira/browse/YARN-4044
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, timelineserver
>Affects Versions: 2.7.0
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Critical
> Attachments: 0001-YARN-4044.patch
>
>
> SystemMetricsPublisher need to expose an appUpdated api to update any change
> for a running application.
> Events can be
> - change of queue for a running application.
> - change of application priority for a running application.
> This ticket intends to handle both RM and timeline side changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4044) Running applications information changes such as movequeue is not published to TimeLine server
[ https://issues.apache.org/jira/browse/YARN-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14682033#comment-14682033 ] Sunil G commented on YARN-4044: --- Timeline v2 changes can be tracked in separate ticket once api changes are done. I will file a ticket under V2 umbrella jira if no issues. > Running applications information changes such as movequeue is not published > to TimeLine server > -- > > Key: YARN-4044 > URL: https://issues.apache.org/jira/browse/YARN-4044 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineserver >Affects Versions: 2.7.0 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > > SystemMetricsPublisher need to expose an appUpdated api to update any change > for a running application. > Events can be > - change of queue for a running application. > - change of application priority for a running application. > This ticket intends to handle both RM and timeline side changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
