[jira] [Commented] (HADOOP-14128) ChecksumFs should override rename with overwrite flag

2018-11-06 Thread Hrishikesh Gadre (JIRA) 


[ 
https://issues.apache.org/jira/browse/HADOOP-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677332#comment-16677332
 ] 

Hrishikesh Gadre commented on HADOOP-14128:
---

[~mathieu_c] given that the last pre-commit run is successful with my patch 
containing your test only changes (but not the actual fix), I am inclined to 
close this jira as "Not a Bug". Please let me know if you have any thoughts 
otherwise.

CC [~xiaochen]

 

> ChecksumFs should override rename with overwrite flag
> -
>
> Key: HADOOP-14128
> URL: https://issues.apache.org/jira/browse/HADOOP-14128
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, fs
>Affects Versions: 2.8.1
>Reporter: Mathieu Chataigner
>Assignee: Hrishikesh Gadre
>Priority: Major
> Attachments: HADOOP-14128-003.patch, 
> HADOOP-14128-branch-2.8.1-001.patch, HADOOP-14128-branch2.8.1-001.patch, 
> HADOOP-14128.001.patch, HADOOP-14128.002.patch
>
>
> When I call FileContext.rename(src, dst, Options.Rename.OVERWRITE) on a 
> LocalFs (which extends ChecksumFs), it does not update crc files.
> Every subsequent read on moved files will result in failures due to crc 
> missmatch.
> One solution is to override rename(src, dst, overwrite) the same way it's 
> done with rename(src, dst) and moving crc files accordingly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-14128) ChecksumFs should override rename with overwrite flag

2018-11-06 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/HADOOP-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677310#comment-16677310
 ] 

Hadoop QA commented on HADOOP-14128:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m 
 1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
14m 13s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
59s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m 
14s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 52s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch 
generated 1 new + 72 unchanged - 0 fixed = 73 total (was 72) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
11m 57s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  8m 
47s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
41s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 99m  1s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-14128 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12947119/HADOOP-14128-003.patch
 |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  |
| uname | Linux fe7cc404c91b 3.13.0-144-generic #193-Ubuntu SMP Thu Mar 15 
17:03:53 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 08d69d9 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/15480/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/15480/testReport/ |
| Max. process+thread count | 1765 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U: 
hadoop-common-project/hadoop-common |
| Console output | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/15480/console |
| Powered by |

[jira] [Commented] (HADOOP-14128) ChecksumFs should override rename with overwrite flag

2018-11-06 Thread Hrishikesh Gadre (JIRA) 


[ 
https://issues.apache.org/jira/browse/HADOOP-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677174#comment-16677174
 ] 

Hrishikesh Gadre commented on HADOOP-14128:
---

[~xiaochen] mentioned that the pre-commit build for branch-2.8 is broken. Let 
me resubmit the patch against the trunk.

> ChecksumFs should override rename with overwrite flag
> -
>
> Key: HADOOP-14128
> URL: https://issues.apache.org/jira/browse/HADOOP-14128
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, fs
>Affects Versions: 2.8.1
>Reporter: Mathieu Chataigner
>Assignee: Hrishikesh Gadre
>Priority: Major
> Attachments: HADOOP-14128-branch-2.8.1-001.patch, 
> HADOOP-14128-branch2.8.1-001.patch, HADOOP-14128.001.patch, 
> HADOOP-14128.002.patch
>
>
> When I call FileContext.rename(src, dst, Options.Rename.OVERWRITE) on a 
> LocalFs (which extends ChecksumFs), it does not update crc files.
> Every subsequent read on moved files will result in failures due to crc 
> missmatch.
> One solution is to override rename(src, dst, overwrite) the same way it's 
> done with rename(src, dst) and moving crc files accordingly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-14128) ChecksumFs should override rename with overwrite flag

2018-11-05 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/HADOOP-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676097#comment-16676097
 ] 

Hadoop QA commented on HADOOP-14128:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red}  6m 
53s{color} | {color:red} Docker failed to build yetus/hadoop:5af2af1. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-14128 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12946999/HADOOP-14128-branch-2.8.1-001.patch
 |
| Console output | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/15473/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> ChecksumFs should override rename with overwrite flag
> -
>
> Key: HADOOP-14128
> URL: https://issues.apache.org/jira/browse/HADOOP-14128
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, fs
>Affects Versions: 2.8.1
>Reporter: Mathieu Chataigner
>Assignee: Hrishikesh Gadre
>Priority: Major
> Attachments: HADOOP-14128-branch-2.8.1-001.patch, 
> HADOOP-14128-branch2.8.1-001.patch, HADOOP-14128.001.patch, 
> HADOOP-14128.002.patch
>
>
> When I call FileContext.rename(src, dst, Options.Rename.OVERWRITE) on a 
> LocalFs (which extends ChecksumFs), it does not update crc files.
> Every subsequent read on moved files will result in failures due to crc 
> missmatch.
> One solution is to override rename(src, dst, overwrite) the same way it's 
> done with rename(src, dst) and moving crc files accordingly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-14128) ChecksumFs should override rename with overwrite flag

2018-11-05 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/HADOOP-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675978#comment-16675978
 ] 

Hadoop QA commented on HADOOP-14128:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red}  3m 
12s{color} | {color:red} Docker failed to build yetus/hadoop:5af2af1. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-14128 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12946999/HADOOP-14128-branch-2.8.1-001.patch
 |
| Console output | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/15471/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> ChecksumFs should override rename with overwrite flag
> -
>
> Key: HADOOP-14128
> URL: https://issues.apache.org/jira/browse/HADOOP-14128
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, fs
>Affects Versions: 2.8.1
>Reporter: Mathieu Chataigner
>Assignee: Hrishikesh Gadre
>Priority: Major
> Attachments: HADOOP-14128-branch-2.8.1-001.patch, 
> HADOOP-14128-branch2.8.1-001.patch, HADOOP-14128.001.patch, 
> HADOOP-14128.002.patch
>
>
> When I call FileContext.rename(src, dst, Options.Rename.OVERWRITE) on a 
> LocalFs (which extends ChecksumFs), it does not update crc files.
> Every subsequent read on moved files will result in failures due to crc 
> missmatch.
> One solution is to override rename(src, dst, overwrite) the same way it's 
> done with rename(src, dst) and moving crc files accordingly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-14128) ChecksumFs should override rename with overwrite flag

2018-11-05 Thread Hrishikesh Gadre (JIRA) 


[ 
https://issues.apache.org/jira/browse/HADOOP-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675977#comment-16675977
 ] 

Hrishikesh Gadre commented on HADOOP-14128:
---

Sorry! File name was incorrect. Uploaded the patch again.

> ChecksumFs should override rename with overwrite flag
> -
>
> Key: HADOOP-14128
> URL: https://issues.apache.org/jira/browse/HADOOP-14128
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, fs
>Affects Versions: 2.8.1
>Reporter: Mathieu Chataigner
>Assignee: Hrishikesh Gadre
>Priority: Major
> Attachments: HADOOP-14128-branch-2.8.1-001.patch, 
> HADOOP-14128-branch2.8.1-001.patch, HADOOP-14128.001.patch, 
> HADOOP-14128.002.patch
>
>
> When I call FileContext.rename(src, dst, Options.Rename.OVERWRITE) on a 
> LocalFs (which extends ChecksumFs), it does not update crc files.
> Every subsequent read on moved files will result in failures due to crc 
> missmatch.
> One solution is to override rename(src, dst, overwrite) the same way it's 
> done with rename(src, dst) and moving crc files accordingly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-14128) ChecksumFs should override rename with overwrite flag

2018-11-05 Thread Hrishikesh Gadre (JIRA) 


[ 
https://issues.apache.org/jira/browse/HADOOP-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675975#comment-16675975
 ] 

Hrishikesh Gadre commented on HADOOP-14128:
---

[~mathieu_c] thanks for reporting the issue and the patch :) I reviewed the 
patch and the relevant functionality in general and I feel like the code is 
working as expected (i.e. the root cause of  of not updating CRC files doesn't 
seem to be related to rename functionality). To confirm this, I have uploaded a 
new patch (HADOOP-14128-branch2.8.1-001.patch) which contains ONLY test changes 
that you proposed in your patch. On my local enviornment, the modified test is 
passing even without your fix in ChecksumFs.java. The only way to to fail this 
modified test was to make renameInternal(src, dest, overwrite) method in 
ChecksumFs.java a NOOP (i.e. not calling method in super class). But since 
ChecksumFs.java doesn't override renameInternal(src, dest, overwrite) method, I 
think renameInternal(src, dest, overwrite) method in AbstractFileSystem class 
is doing the right thing.

Any thoughts? It would be nice to see the test results of my patch as well.

> ChecksumFs should override rename with overwrite flag
> -
>
> Key: HADOOP-14128
> URL: https://issues.apache.org/jira/browse/HADOOP-14128
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, fs
>Affects Versions: 2.8.1
>Reporter: Mathieu Chataigner
>Assignee: Hrishikesh Gadre
>Priority: Major
> Attachments: HADOOP-14128-branch2.8.1-001.patch, 
> HADOOP-14128.001.patch, HADOOP-14128.002.patch
>
>
> When I call FileContext.rename(src, dst, Options.Rename.OVERWRITE) on a 
> LocalFs (which extends ChecksumFs), it does not update crc files.
> Every subsequent read on moved files will result in failures due to crc 
> missmatch.
> One solution is to override rename(src, dst, overwrite) the same way it's 
> done with rename(src, dst) and moving crc files accordingly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-14128) ChecksumFs should override rename with overwrite flag

2018-11-05 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/HADOOP-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675968#comment-16675968
 ] 

Hadoop QA commented on HADOOP-14128:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  5s{color} 
| {color:red} HADOOP-14128 does not apply to trunk. Rebase required? Wrong 
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-14128 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12946998/HADOOP-14128-branch2.8.1-001.patch
 |
| Console output | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/15470/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> ChecksumFs should override rename with overwrite flag
> -
>
> Key: HADOOP-14128
> URL: https://issues.apache.org/jira/browse/HADOOP-14128
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, fs
>Affects Versions: 2.8.1
>Reporter: Mathieu Chataigner
>Assignee: Hrishikesh Gadre
>Priority: Major
> Attachments: HADOOP-14128-branch2.8.1-001.patch, 
> HADOOP-14128.001.patch, HADOOP-14128.002.patch
>
>
> When I call FileContext.rename(src, dst, Options.Rename.OVERWRITE) on a 
> LocalFs (which extends ChecksumFs), it does not update crc files.
> Every subsequent read on moved files will result in failures due to crc 
> missmatch.
> One solution is to override rename(src, dst, overwrite) the same way it's 
> done with rename(src, dst) and moving crc files accordingly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-14128) ChecksumFs should override rename with overwrite flag

2018-10-31 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/HADOOP-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670733#comment-16670733
 ] 

Hadoop QA commented on HADOOP-14128:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m 
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
14m  6s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
0s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m 
33s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 52s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch 
generated 2 new + 102 unchanged - 0 fixed = 104 total (was 102) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
11m 46s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  8m 
30s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
39s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}102m 36s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-14128 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12896927/HADOOP-14128.002.patch
 |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  |
| uname | Linux b752ee4a6ead 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 
14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 6668c19 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/15435/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/15435/testReport/ |
| Max. process+thread count | 1579 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U: 
hadoop-common-project/hadoop-common |
| Console output | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/15435/console |
| Powered by |

[jira] [Commented] (HADOOP-14128) ChecksumFs should override rename with overwrite flag

2017-11-09 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246631#comment-16246631
 ] 

Hadoop QA commented on HADOOP-14128:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  9m 
32s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m 
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m  
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
12m  4s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
59s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m  
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m  
1s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 42s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch 
generated 2 new + 102 unchanged - 0 fixed = 104 total (was 102) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}  
9m 36s{color} | {color:green} patch has no errors when building and testing our 
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  9m 48s{color} 
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
33s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}103m 54s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ha.TestZKFailoverController |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | HADOOP-14128 |
| GITHUB PR | https://github.com/apache/hadoop/pull/290 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  shadedclient  findbugs  checkstyle  |
| uname | Linux b632ecd1abab 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19 
15:49:21 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / a2c150a |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/13657/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/13657/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/13657/testReport/ |
| Max. process+thread count | 1432 (vs. ulimit of 5000) |
| modules | C:

[jira] [Commented] (HADOOP-14128) ChecksumFs should override rename with overwrite flag

2017-11-09 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246422#comment-16246422
 ] 

Hadoop QA commented on HADOOP-14128:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 15m 
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m 
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m 
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
11m 34s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
54s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m  
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m  
6s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 37s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch 
generated 2 new + 102 unchanged - 0 fixed = 104 total (was 102) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}  
9m 54s{color} | {color:green} patch has no errors when building and testing our 
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  8m 
51s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
31s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 97m 16s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | HADOOP-14128 |
| GITHUB PR | https://github.com/apache/hadoop/pull/290 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  shadedclient  findbugs  checkstyle  |
| uname | Linux 5bae131bdb5d 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 6c32dda |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/13655/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/13655/testReport/ |
| Max. process+thread count | 1333 (vs. ulimit of 5000) |
| modules | C: hadoop-common-project/hadoop-common U: 
hadoop-common-project/hadoop-common |
| Console output | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/13655/console |
| Powered by | Apache Yetus 0.7.0-SNAPSHOT   http://yetus.apache.org |

[jira] [Commented] (HADOOP-14128) ChecksumFs should override rename with overwrite flag

2017-11-09 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246215#comment-16246215
 ] 

ASF GitHub Bot commented on HADOOP-14128:
-

GitHub user mchataigner opened a pull request:

https://github.com/apache/hadoop/pull/290

HADOOP-14128. fix renameInternal in ChecksumFs

AbstractFs.rename(source, destination, options) calls
renameInternal(source, destination, overwrite)

This patch adds this method to ChecksumFs to rename the crc file in
addition to the file itself to avoid crc missmatch when use for example
in LocalFs.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mchataigner/hadoop fix_checksumfs

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hadoop/pull/290.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #290


commit b31cd6882ca89b18bb229ef108022026d284c70a
Author: Mathieu Chataigner 
Date:   2017-11-09T18:23:48Z

HADOOP-14128. fix renameInternal in ChecksumFs

AbstractFs.rename(source, destination, options) calls
renameInternal(source, destination, overwrite)

This patch adds this method to ChecksumFs to rename the crc file in
addition to the file itself to avoid crc missmatch when use for example
in LocalFs.




> ChecksumFs should override rename with overwrite flag
> -
>
> Key: HADOOP-14128
> URL: https://issues.apache.org/jira/browse/HADOOP-14128
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, fs
>Reporter: Mathieu Chataigner
> Attachments: HADOOP-14128.001.patch
>
>
> When I call FileContext.rename(src, dst, Options.Rename.OVERWRITE) on a 
> LocalFs (which extends ChecksumFs), it does not update crc files.
> Every subsequent read on moved files will result in failures due to crc 
> missmatch.
> One solution is to override rename(src, dst, overwrite) the same way it's 
> done with rename(src, dst) and moving crc files accordingly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-14128) ChecksumFs should override rename with overwrite flag

2017-02-27 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15886733#comment-15886733
 ] 

Hadoop QA commented on HADOOP-14128:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m 
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m 
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m  
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m  
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  8m 36s{color} 
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
32s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 59m  4s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestKDiag |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-14128 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12854993/HADOOP-14128.001.patch
 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux 53a9c2a9fb27 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / 5f5b031 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/11722/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/11722/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U: 
hadoop-common-project/hadoop-common |
| Console output | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/11722/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> ChecksumFs should override rename with overwrite flag
> -
>
> Key: HADOOP-14128
> URL: https://issues.apache.org/jira/browse/HADOOP-14128
> Project: Hadoop Common
>  Issue