[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178497#comment-14178497 ] Hudson commented on YARN-2582: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1933 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1933/]) YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong. (zjshen: rev e90718fa5a0e7c18592af61534668acebb9db51b) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogCLIHelpers.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/LogsCLI.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestLogsCLI.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/logaggregation/TestAggregatedLogsBlock.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/log/AggregatedLogsBlock.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogAggregationUtils.java * hadoop-yarn-project/CHANGES.txt > Log related CLI and Web UI changes for Aggregated Logs in LRS > - > > Key: YARN-2582 > URL: https://issues.apache.org/jira/browse/YARN-2582 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Xuan Gong >Assignee: Xuan Gong > Fix For: 2.6.0 > > Attachments: YARN-2582.1.patch, YARN-2582.2.patch, YARN-2582.3.patch > > > After YARN-2468, we have change the log layout to support log aggregation for > Long Running Service. Log CLI and related Web UI should be modified > accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178427#comment-14178427 ] Hudson commented on YARN-2582: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1908 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1908/]) YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong. (zjshen: rev e90718fa5a0e7c18592af61534668acebb9db51b) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogCLIHelpers.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/LogsCLI.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestLogsCLI.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/logaggregation/TestAggregatedLogsBlock.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogAggregationUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/log/AggregatedLogsBlock.java > Log related CLI and Web UI changes for Aggregated Logs in LRS > - > > Key: YARN-2582 > URL: https://issues.apache.org/jira/browse/YARN-2582 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Xuan Gong >Assignee: Xuan Gong > Fix For: 2.6.0 > > Attachments: YARN-2582.1.patch, YARN-2582.2.patch, YARN-2582.3.patch > > > After YARN-2468, we have change the log layout to support log aggregation for > Long Running Service. Log CLI and related Web UI should be modified > accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178308#comment-14178308 ] Hudson commented on YARN-2582: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #719 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/719/]) YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong. (zjshen: rev e90718fa5a0e7c18592af61534668acebb9db51b) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/log/AggregatedLogsBlock.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogCLIHelpers.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/LogsCLI.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestLogsCLI.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/logaggregation/TestAggregatedLogsBlock.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogAggregationUtils.java > Log related CLI and Web UI changes for Aggregated Logs in LRS > - > > Key: YARN-2582 > URL: https://issues.apache.org/jira/browse/YARN-2582 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Xuan Gong >Assignee: Xuan Gong > Fix For: 2.6.0 > > Attachments: YARN-2582.1.patch, YARN-2582.2.patch, YARN-2582.3.patch > > > After YARN-2468, we have change the log layout to support log aggregation for > Long Running Service. Log CLI and related Web UI should be modified > accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177464#comment-14177464 ] Hudson commented on YARN-2582: -- FAILURE: Integrated in Hadoop-trunk-Commit #6294 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6294/]) YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong. (zjshen: rev e90718fa5a0e7c18592af61534668acebb9db51b) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogAggregationUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogCLIHelpers.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/logaggregation/TestAggregatedLogsBlock.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/log/AggregatedLogsBlock.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestLogsCLI.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/LogsCLI.java > Log related CLI and Web UI changes for Aggregated Logs in LRS > - > > Key: YARN-2582 > URL: https://issues.apache.org/jira/browse/YARN-2582 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Xuan Gong >Assignee: Xuan Gong > Fix For: 2.6.0 > > Attachments: YARN-2582.1.patch, YARN-2582.2.patch, YARN-2582.3.patch > > > After YARN-2468, we have change the log layout to support log aggregation for > Long Running Service. Log CLI and related Web UI should be modified > accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177392#comment-14177392 ] Zhijie Shen commented on YARN-2582: --- +1 for the last patch. Will commit it. > Log related CLI and Web UI changes for Aggregated Logs in LRS > - > > Key: YARN-2582 > URL: https://issues.apache.org/jira/browse/YARN-2582 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-2582.1.patch, YARN-2582.2.patch, YARN-2582.3.patch > > > After YARN-2468, we have change the log layout to support log aggregation for > Long Running Service. Log CLI and related Web UI should be modified > accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177366#comment-14177366 ] Hadoop QA commented on YARN-2582: - {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12675907/YARN-2582.3.patch against trunk revision d5084b9. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5463//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5463//console This message is automatically generated. > Log related CLI and Web UI changes for Aggregated Logs in LRS > - > > Key: YARN-2582 > URL: https://issues.apache.org/jira/browse/YARN-2582 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-2582.1.patch, YARN-2582.2.patch, YARN-2582.3.patch > > > After YARN-2468, we have change the log layout to support log aggregation for > Long Running Service. Log CLI and related Web UI should be modified > accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177296#comment-14177296 ] Xuan Gong commented on YARN-2582: - Thanks for the review. Uploaded a new patch to address all the comments > Log related CLI and Web UI changes for Aggregated Logs in LRS > - > > Key: YARN-2582 > URL: https://issues.apache.org/jira/browse/YARN-2582 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-2582.1.patch, YARN-2582.2.patch, YARN-2582.3.patch > > > After YARN-2468, we have change the log layout to support log aggregation for > Long Running Service. Log CLI and related Web UI should be modified > accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177274#comment-14177274 ] Zhijie Shen commented on YARN-2582: --- Looks good to me overall. Just one nit Make the following methods static? {code} private void containerLogNotFound(String containerId) { System.out.println("Logs for container " + containerId + " are not present in this log-file."); } private void logDirNotExist(String remoteAppLogDir) { System.out.println(remoteAppLogDir + "does not exist."); System.out.println("Log aggregation has not completed or is not enabled."); } private void emptyLogDir(String remoteAppLogDir) { System.out.println(remoteAppLogDir + "does not have any log files."); } {code} Same for {code} private void createContainerLogInLocalDir(Path appLogsDir, ContainerId containerId, FileSystem fs) throws Exception { {code} and {code} private void uploadContainerLogIntoRemoteDir(UserGroupInformation ugi, Configuration configuration, List rootLogDirs, NodeId nodeId, ContainerId containerId, Path appDir, FileSystem fs) throws Exception { {code} > Log related CLI and Web UI changes for Aggregated Logs in LRS > - > > Key: YARN-2582 > URL: https://issues.apache.org/jira/browse/YARN-2582 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-2582.1.patch, YARN-2582.2.patch > > > After YARN-2468, we have change the log layout to support log aggregation for > Long Running Service. Log CLI and related Web UI should be modified > accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14175343#comment-14175343 ] Xuan Gong commented on YARN-2582: - The CLI and Web UI changes for aggregated logs will be fixed in this ticket. The CLI and Web UI changes for running applications' logs will be fixed separately. Those will be tracked by YARN-2658 and YARN-2659. Also, we could improve the log display be adding loguploadtime into logValue content. This will be tracked by YARN-2703 > Log related CLI and Web UI changes for Aggregated Logs in LRS > - > > Key: YARN-2582 > URL: https://issues.apache.org/jira/browse/YARN-2582 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-2582.1.patch, YARN-2582.2.patch > > > After YARN-2468, we have change the log layout to support log aggregation for > Long Running Service. Log CLI and related Web UI should be modified > accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169788#comment-14169788 ] Hadoop QA commented on YARN-2582: - {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12674019/YARN-2582.2.patch against trunk revision a56ea01. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5377//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5377//console This message is automatically generated. > Log related CLI and Web UI changes for Aggregated Logs in LRS > - > > Key: YARN-2582 > URL: https://issues.apache.org/jira/browse/YARN-2582 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-2582.1.patch, YARN-2582.2.patch > > > After YARN-2468, we have change the log layout to support log aggregation for > Long Running Service. Log CLI and related Web UI should be modified > accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14165926#comment-14165926 ] Xuan Gong commented on YARN-2582: - -1 release audit is unrelated > Log related CLI and Web UI changes for Aggregated Logs in LRS > - > > Key: YARN-2582 > URL: https://issues.apache.org/jira/browse/YARN-2582 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-2582.1.patch, YARN-2582.2.patch > > > After YARN-2468, we have change the log layout to support log aggregation for > Long Running Service. Log CLI and related Web UI should be modified > accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14165920#comment-14165920 ] Hadoop QA commented on YARN-2582: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12674019/YARN-2582.2.patch against trunk revision 8d94114. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:red}-1 release audit{color}. The applied patch generated 1 release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5347//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-YARN-Build/5347//artifact/patchprocess/patchReleaseAuditProblems.txt Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5347//console This message is automatically generated. > Log related CLI and Web UI changes for Aggregated Logs in LRS > - > > Key: YARN-2582 > URL: https://issues.apache.org/jira/browse/YARN-2582 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-2582.1.patch, YARN-2582.2.patch > > > After YARN-2468, we have change the log layout to support log aggregation for > Long Running Service. Log CLI and related Web UI should be modified > accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)