[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14178427#comment-14178427 ] Hudson commented on YARN-2582: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1908 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1908/]) YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong. (zjshen: rev e90718fa5a0e7c18592af61534668acebb9db51b) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogCLIHelpers.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/LogsCLI.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestLogsCLI.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/logaggregation/TestAggregatedLogsBlock.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogAggregationUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/log/AggregatedLogsBlock.java Log related CLI and Web UI changes for Aggregated Logs in LRS - Key: YARN-2582 URL: https://issues.apache.org/jira/browse/YARN-2582 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager, resourcemanager Reporter: Xuan Gong Assignee: Xuan Gong Fix For: 2.6.0 Attachments: YARN-2582.1.patch, YARN-2582.2.patch, YARN-2582.3.patch After YARN-2468, we have change the log layout to support log aggregation for Long Running Service. Log CLI and related Web UI should be modified accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14178497#comment-14178497 ] Hudson commented on YARN-2582: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1933 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1933/]) YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong. (zjshen: rev e90718fa5a0e7c18592af61534668acebb9db51b) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogCLIHelpers.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/LogsCLI.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestLogsCLI.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/logaggregation/TestAggregatedLogsBlock.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/log/AggregatedLogsBlock.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogAggregationUtils.java * hadoop-yarn-project/CHANGES.txt Log related CLI and Web UI changes for Aggregated Logs in LRS - Key: YARN-2582 URL: https://issues.apache.org/jira/browse/YARN-2582 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager, resourcemanager Reporter: Xuan Gong Assignee: Xuan Gong Fix For: 2.6.0 Attachments: YARN-2582.1.patch, YARN-2582.2.patch, YARN-2582.3.patch After YARN-2468, we have change the log layout to support log aggregation for Long Running Service. Log CLI and related Web UI should be modified accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[
https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177274#comment-14177274
]
Zhijie Shen commented on YARN-2582:
---
Looks good to me overall. Just one nit
Make the following methods static?
{code}
private void containerLogNotFound(String containerId) {
System.out.println(Logs for container + containerId
+ are not present in this log-file.);
}
private void logDirNotExist(String remoteAppLogDir) {
System.out.println(remoteAppLogDir + does not exist.);
System.out.println(Log aggregation has not completed or is not enabled.);
}
private void emptyLogDir(String remoteAppLogDir) {
System.out.println(remoteAppLogDir + does not have any log files.);
}
{code}
Same for
{code}
private void createContainerLogInLocalDir(Path appLogsDir,
ContainerId containerId, FileSystem fs) throws Exception {
{code}
and
{code}
private void uploadContainerLogIntoRemoteDir(UserGroupInformation ugi,
Configuration configuration, ListString rootLogDirs, NodeId nodeId,
ContainerId containerId, Path appDir, FileSystem fs) throws Exception {
{code}
Log related CLI and Web UI changes for Aggregated Logs in LRS
-
Key: YARN-2582
URL: https://issues.apache.org/jira/browse/YARN-2582
Project: Hadoop YARN
Issue Type: Sub-task
Components: nodemanager, resourcemanager
Reporter: Xuan Gong
Assignee: Xuan Gong
Attachments: YARN-2582.1.patch, YARN-2582.2.patch
After YARN-2468, we have change the log layout to support log aggregation for
Long Running Service. Log CLI and related Web UI should be modified
accordingly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177296#comment-14177296 ] Xuan Gong commented on YARN-2582: - Thanks for the review. Uploaded a new patch to address all the comments Log related CLI and Web UI changes for Aggregated Logs in LRS - Key: YARN-2582 URL: https://issues.apache.org/jira/browse/YARN-2582 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager, resourcemanager Reporter: Xuan Gong Assignee: Xuan Gong Attachments: YARN-2582.1.patch, YARN-2582.2.patch, YARN-2582.3.patch After YARN-2468, we have change the log layout to support log aggregation for Long Running Service. Log CLI and related Web UI should be modified accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[
https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177366#comment-14177366
]
Hadoop QA commented on YARN-2582:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12675907/YARN-2582.3.patch
against trunk revision d5084b9.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5463//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5463//console
This message is automatically generated.
Log related CLI and Web UI changes for Aggregated Logs in LRS
-
Key: YARN-2582
URL: https://issues.apache.org/jira/browse/YARN-2582
Project: Hadoop YARN
Issue Type: Sub-task
Components: nodemanager, resourcemanager
Reporter: Xuan Gong
Assignee: Xuan Gong
Attachments: YARN-2582.1.patch, YARN-2582.2.patch, YARN-2582.3.patch
After YARN-2468, we have change the log layout to support log aggregation for
Long Running Service. Log CLI and related Web UI should be modified
accordingly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177392#comment-14177392 ] Zhijie Shen commented on YARN-2582: --- +1 for the last patch. Will commit it. Log related CLI and Web UI changes for Aggregated Logs in LRS - Key: YARN-2582 URL: https://issues.apache.org/jira/browse/YARN-2582 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager, resourcemanager Reporter: Xuan Gong Assignee: Xuan Gong Attachments: YARN-2582.1.patch, YARN-2582.2.patch, YARN-2582.3.patch After YARN-2468, we have change the log layout to support log aggregation for Long Running Service. Log CLI and related Web UI should be modified accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177464#comment-14177464 ] Hudson commented on YARN-2582: -- FAILURE: Integrated in Hadoop-trunk-Commit #6294 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6294/]) YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong. (zjshen: rev e90718fa5a0e7c18592af61534668acebb9db51b) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogAggregationUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogCLIHelpers.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/logaggregation/TestAggregatedLogsBlock.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/log/AggregatedLogsBlock.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestLogsCLI.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/LogsCLI.java Log related CLI and Web UI changes for Aggregated Logs in LRS - Key: YARN-2582 URL: https://issues.apache.org/jira/browse/YARN-2582 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager, resourcemanager Reporter: Xuan Gong Assignee: Xuan Gong Fix For: 2.6.0 Attachments: YARN-2582.1.patch, YARN-2582.2.patch, YARN-2582.3.patch After YARN-2468, we have change the log layout to support log aggregation for Long Running Service. Log CLI and related Web UI should be modified accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14175343#comment-14175343 ] Xuan Gong commented on YARN-2582: - The CLI and Web UI changes for aggregated logs will be fixed in this ticket. The CLI and Web UI changes for running applications' logs will be fixed separately. Those will be tracked by YARN-2658 and YARN-2659. Also, we could improve the log display be adding loguploadtime into logValue content. This will be tracked by YARN-2703 Log related CLI and Web UI changes for Aggregated Logs in LRS - Key: YARN-2582 URL: https://issues.apache.org/jira/browse/YARN-2582 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager, resourcemanager Reporter: Xuan Gong Assignee: Xuan Gong Attachments: YARN-2582.1.patch, YARN-2582.2.patch After YARN-2468, we have change the log layout to support log aggregation for Long Running Service. Log CLI and related Web UI should be modified accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[
https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169788#comment-14169788
]
Hadoop QA commented on YARN-2582:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12674019/YARN-2582.2.patch
against trunk revision a56ea01.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5377//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5377//console
This message is automatically generated.
Log related CLI and Web UI changes for Aggregated Logs in LRS
-
Key: YARN-2582
URL: https://issues.apache.org/jira/browse/YARN-2582
Project: Hadoop YARN
Issue Type: Sub-task
Components: nodemanager, resourcemanager
Reporter: Xuan Gong
Assignee: Xuan Gong
Attachments: YARN-2582.1.patch, YARN-2582.2.patch
After YARN-2468, we have change the log layout to support log aggregation for
Long Running Service. Log CLI and related Web UI should be modified
accordingly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[
https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165920#comment-14165920
]
Hadoop QA commented on YARN-2582:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12674019/YARN-2582.2.patch
against trunk revision 8d94114.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5347//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/5347//artifact/patchprocess/patchReleaseAuditProblems.txt
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5347//console
This message is automatically generated.
Log related CLI and Web UI changes for Aggregated Logs in LRS
-
Key: YARN-2582
URL: https://issues.apache.org/jira/browse/YARN-2582
Project: Hadoop YARN
Issue Type: Sub-task
Components: nodemanager, resourcemanager
Reporter: Xuan Gong
Assignee: Xuan Gong
Attachments: YARN-2582.1.patch, YARN-2582.2.patch
After YARN-2468, we have change the log layout to support log aggregation for
Long Running Service. Log CLI and related Web UI should be modified
accordingly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2582) Log related CLI and Web UI changes for Aggregated Logs in LRS
[ https://issues.apache.org/jira/browse/YARN-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165926#comment-14165926 ] Xuan Gong commented on YARN-2582: - -1 release audit is unrelated Log related CLI and Web UI changes for Aggregated Logs in LRS - Key: YARN-2582 URL: https://issues.apache.org/jira/browse/YARN-2582 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager, resourcemanager Reporter: Xuan Gong Assignee: Xuan Gong Attachments: YARN-2582.1.patch, YARN-2582.2.patch After YARN-2468, we have change the log layout to support log aggregation for Long Running Service. Log CLI and related Web UI should be modified accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
