[jira] [Commented] (YARN-7174) Add retry logic in LogsCLI when fetch running application logs
[ https://issues.apache.org/jira/browse/YARN-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168647#comment-16168647 ] Hudson commented on YARN-7174: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12890 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12890/]) YARN-7174. Add retry logic in LogsCLI when fetch running application (junping_du: rev 1a84c24b0cf6674fa755403971fa57d8e412b320) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestLogsCLI.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/LogsCLI.java > Add retry logic in LogsCLI when fetch running application logs > -- > > Key: YARN-7174 > URL: https://issues.apache.org/jira/browse/YARN-7174 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Fix For: 2.9.0, 3.0.0-beta1, 3.1.0 > > Attachments: YARN-7174.1.patch, YARN-7174.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7174) Add retry logic in LogsCLI when fetch running application logs
[ https://issues.apache.org/jira/browse/YARN-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168542#comment-16168542 ] Junping Du commented on YARN-7174: -- got it. v2 patch LGTM. +1. Will commit it shortly. > Add retry logic in LogsCLI when fetch running application logs > -- > > Key: YARN-7174 > URL: https://issues.apache.org/jira/browse/YARN-7174 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7174.1.patch, YARN-7174.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7174) Add retry logic in LogsCLI when fetch running application logs
[ https://issues.apache.org/jira/browse/YARN-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16166976#comment-16166976 ] Xuan Gong commented on YARN-7174: - bq. I think we haven't addressed this comment in v2 patch? I have included those in v2 patch. But because we have added a new option, the format of the help messages changed a little bit. > Add retry logic in LogsCLI when fetch running application logs > -- > > Key: YARN-7174 > URL: https://issues.apache.org/jira/browse/YARN-7174 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7174.1.patch, YARN-7174.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7174) Add retry logic in LogsCLI when fetch running application logs
[ https://issues.apache.org/jira/browse/YARN-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16166895#comment-16166895 ] Junping Du commented on YARN-7174: -- Thanks [~xgong] for updating the patch. The patch looks good to me in overall. Just one small issue: -- Looks like we remove the verification of output of yarn log cli usage. Any special reason to do this? If not, we should add it back to prevent careless/unexpected change of usage output. Make sense. Add them back. I think we haven't addressed this comment in v2 patch? > Add retry logic in LogsCLI when fetch running application logs > -- > > Key: YARN-7174 > URL: https://issues.apache.org/jira/browse/YARN-7174 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7174.1.patch, YARN-7174.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7174) Add retry logic in LogsCLI when fetch running application logs
[
https://issues.apache.org/jira/browse/YARN-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16166814#comment-16166814
]
Hadoop QA commented on YARN-7174:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 13s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 80 new
+ 43 unchanged - 60 fixed = 123 total (was 103) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 47s{color}
| {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 28s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.client.api.impl.TestAMRMClientContainerRequest |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7174 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887149/YARN-7174.2.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux c8b54dc9977c 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 66ca0a6 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/17458/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/17458/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17458/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17458/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add retry logic in LogsCLI when fetch running application logs
>
[jira] [Commented] (YARN-7174) Add retry logic in LogsCLI when fetch running application logs
[ https://issues.apache.org/jira/browse/YARN-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16166745#comment-16166745 ] Xuan Gong commented on YARN-7174: - Thanks for the review. [~djp] bq. Looks like we remove the verification of output of yarn log cli usage. Any special reason to do this? If not, we should add it back to prevent careless/unexpected change of usage output. Make sense. Add them back bq. Sounds like we were defining the default max retries and retry intervals inline code, but better separated it out - may be too heavy to define in YarnConfiguration but at least define it in front of class. DONE. bq. After adding two options to log CLI, we should add documentation change in yarn log CLI page Looks like that we missed many more options for log cli. Let us do it separately. https://issues.apache.org/jira/browse/YARN-7195 > Add retry logic in LogsCLI when fetch running application logs > -- > > Key: YARN-7174 > URL: https://issues.apache.org/jira/browse/YARN-7174 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7174.1.patch, YARN-7174.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7174) Add retry logic in LogsCLI when fetch running application logs
[
https://issues.apache.org/jira/browse/YARN-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165395#comment-16165395
]
Junping Du commented on YARN-7174:
--
Thanks [~xgong] for reporting the issue and delivering a fix for it. I quickly
go through the patch, a couple of comments:
{noformat}
+int maxRetries = 30;
+long retryInterval = 1000;
{noformat}
Sounds like we were defining the default max retries and retry intervals inline
code, but better separated it out - may be too heavy to define in
YarnConfiguration but at least define it in front of class.
After adding two options to log CLI, we should add documentation change in yarn
log CLI page
(https://hadoop.apache.org/docs/r2.8.0/hadoop-yarn/hadoop-yarn-site/YarnCommands.html#logs).
Looks like we remove the verification of output of yarn log cli usage. Any
special reason to do this? If not, we should add it back to prevent
careless/unexpected change of usage output.
> Add retry logic in LogsCLI when fetch running application logs
> --
>
> Key: YARN-7174
> URL: https://issues.apache.org/jira/browse/YARN-7174
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: YARN-7174.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7174) Add retry logic in LogsCLI when fetch running application logs
[
https://issues.apache.org/jira/browse/YARN-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157660#comment-16157660
]
Hadoop QA commented on YARN-7174:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 13s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 4 new +
43 unchanged - 60 fixed = 47 total (was 103) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 20m
56s{color} | {color:green} hadoop-yarn-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7174 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12885899/YARN-7174.1.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 08be7ca04bc4 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 83449ab |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/17339/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17339/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17339/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add retry logic in LogsCLI when fetch running application logs
> --
>
> Key: YARN-7174
> URL: https://issues.apache.org/jira/browse/YARN-7174
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Xuan Gong
>
[jira] [Commented] (YARN-7174) Add retry logic in LogsCLI when fetch running application logs
[ https://issues.apache.org/jira/browse/YARN-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157587#comment-16157587 ] Xuan Gong commented on YARN-7174: - Uploaded a patch for this. Note that I have modified the testHelpMessage() which would directly test that the help message would include all the command options. So, we do not need to manually add newly helper messages. > Add retry logic in LogsCLI when fetch running application logs > -- > > Key: YARN-7174 > URL: https://issues.apache.org/jira/browse/YARN-7174 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7174.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
