[jira] [Commented] (YARN-7194) Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat

2017-10-02 Thread Xuan Gong (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-7194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189060#comment-16189060
 ] 

Xuan Gong commented on YARN-7194:
-

pushed to branch-3.0 as well

> Log aggregation status is always Failed with the newly added log aggregation 
> IndexedFileFormat
> --
>
> Key: YARN-7194
> URL: https://issues.apache.org/jira/browse/YARN-7194
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Fix For: 2.9.0, 3.1.0
>
> Attachments: YARN-7194.1.patch, YARN-7194.2.patch, YARN-7194.3.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-7194) Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat

2017-09-13 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-7194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165772#comment-16165772
 ] 

Hudson commented on YARN-7194:
--

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12869 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/12869/])
YARN-7194. Log aggregation status is always Failed with the newly added 
(junping_du: rev c92c1d521eadfd8a4cd8205cc6aee74816f353f4)
* (edit) 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifile/LogAggregationIndexedFileController.java
* (edit) 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifile/TestLogAggregationIndexFileController.java


> Log aggregation status is always Failed with the newly added log aggregation 
> IndexedFileFormat
> --
>
> Key: YARN-7194
> URL: https://issues.apache.org/jira/browse/YARN-7194
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Fix For: 2.9.0, 3.1.0
>
> Attachments: YARN-7194.1.patch, YARN-7194.2.patch, YARN-7194.3.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-7194) Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat

2017-09-13 Thread Junping Du (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-7194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165764#comment-16165764
 ] 

Junping Du commented on YARN-7194:
--

I have commit the patch to trunk and branch-2. Thanks [~xgong] for patch 
contribution!

> Log aggregation status is always Failed with the newly added log aggregation 
> IndexedFileFormat
> --
>
> Key: YARN-7194
> URL: https://issues.apache.org/jira/browse/YARN-7194
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Fix For: 2.9.0, 3.1.0
>
> Attachments: YARN-7194.1.patch, YARN-7194.2.patch, YARN-7194.3.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-7194) Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat

2017-09-13 Thread Junping Du (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-7194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165755#comment-16165755
 ] 

Junping Du commented on YARN-7194:
--

Thanks [~xgong] for fixing this issue. +1. Committing it in.

> Log aggregation status is always Failed with the newly added log aggregation 
> IndexedFileFormat
> --
>
> Key: YARN-7194
> URL: https://issues.apache.org/jira/browse/YARN-7194
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: YARN-7194.1.patch, YARN-7194.2.patch, YARN-7194.3.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-7194) Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat

2017-09-13 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-7194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165664#comment-16165664
 ] 

Hadoop QA commented on YARN-7194:
-

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m 
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
43s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
49s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
17s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 17s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7194 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12887017/YARN-7194.3.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux a64670c63e69 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / e0b3c64 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/17451/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common |
| Console output | 
https://builds.apache.org/job/PreCommit-YARN-Build/17451/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Log aggregation status is always Failed with the newly added log aggregation 
> IndexedFileFormat
> --
>
> Key: YARN-7194
> URL: https://issues.apache.org/jira/browse/YARN-7194
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: YARN-7194.1.patch, YARN-7194.2.patch, YARN-7194.3.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To

[jira] [Commented] (YARN-7194) Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat

2017-09-13 Thread Xuan Gong (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-7194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165643#comment-16165643
 ] 

Xuan Gong commented on YARN-7194:
-

Fix the testcase failure.

> Log aggregation status is always Failed with the newly added log aggregation 
> IndexedFileFormat
> --
>
> Key: YARN-7194
> URL: https://issues.apache.org/jira/browse/YARN-7194
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: YARN-7194.1.patch, YARN-7194.2.patch, YARN-7194.3.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-7194) Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat

2017-09-13 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-7194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165603#comment-16165603
 ] 

Hadoop QA commented on YARN-7194:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m 
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
42s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  1m 
29s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common 
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  2m 36s{color} 
| {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 16s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common |
|  |  Potentially dangerous use of non-short-circuit logic in 
org.apache.hadoop.yarn.logaggregation.filecontroller.ifile.LogAggregationIndexedFileController.postWrite(LogAggregationFileControllerContext)
  At LogAggregationIndexedFileController.java:logic in 
org.apache.hadoop.yarn.logaggregation.filecontroller.ifile.LogAggregationIndexedFileController.postWrite(LogAggregationFileControllerContext)
  At LogAggregationIndexedFileController.java:[line 396] |
| Failed junit tests | 
hadoop.yarn.logaggregation.filecontroller.ifile.TestLogAggregationIndexFileController
 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7194 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12887001/YARN-7194.1.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux 25786756f7d4 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / 73aed34 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| findbugs | 
https://builds.apache.org/job/PreCommit-YARN-Build/17448/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html
 |
| unit |

[jira] [Commented] (YARN-7194) Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat

2017-09-13 Thread Xuan Gong (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-7194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165576#comment-16165576
 ] 

Xuan Gong commented on YARN-7194:
-

Trivial fix without testcases

> Log aggregation status is always Failed with the newly added log aggregation 
> IndexedFileFormat
> --
>
> Key: YARN-7194
> URL: https://issues.apache.org/jira/browse/YARN-7194
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: YARN-7194.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-7194) Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat

2017-09-13 Thread Xuan Gong (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-7194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165572#comment-16165572
 ] 

Xuan Gong commented on YARN-7194:
-

Uploaded a simple fix:
When we are not in partial log aggregation, we would not try to delete the 
checksum file in postWrite

> Log aggregation status is always Failed with the newly added log aggregation 
> IndexedFileFormat
> --
>
> Key: YARN-7194
> URL: https://issues.apache.org/jira/browse/YARN-7194
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Xuan Gong
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-7194) Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat

2017-09-13 Thread Xuan Gong (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-7194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165571#comment-16165571
 ] 

Xuan Gong commented on YARN-7194:
-

It happens when we are *not* in partial log aggregation.  
When we are not in partial log aggregation, we will not create intermediate 
checksum file, so, when we do postWrite, we would call
{code}
deleteFileWithRetries(fc, ugi, remoteLogCheckSumFile);
{code}
which will throw out exception, and complain that the remoteLogCheckSumFile can 
not be found. So, we would mark the log aggregation status as Failed when we 
catch the exception.

> Log aggregation status is always Failed with the newly added log aggregation 
> IndexedFileFormat
> --
>
> Key: YARN-7194
> URL: https://issues.apache.org/jira/browse/YARN-7194
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Xuan Gong
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org