[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14945781#comment-14945781 ] Hudson commented on HDFS-9180: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #461 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/461/]) HDFS-9180. Update excluded DataNodes in DFSStripedOutputStream based on (jing9: rev a8b4d0ff283a0af1075aaa94904d4c6e63a9a3dd) * hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/StripedFileTestUtil.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestWriteReadStripedFile.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReadStripedFileWithDecoding.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripedDataStreamer.java > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Fix For: 3.0.0 > > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch, HDFS-9180.003.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14945754#comment-14945754 ] Hudson commented on HDFS-9180: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #1225 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1225/]) HDFS-9180. Update excluded DataNodes in DFSStripedOutputStream based on (jing9: rev a8b4d0ff283a0af1075aaa94904d4c6e63a9a3dd) * hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripedDataStreamer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/StripedFileTestUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestWriteReadStripedFile.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReadStripedFileWithDecoding.java > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Fix For: 3.0.0 > > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch, HDFS-9180.003.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14945728#comment-14945728 ] Hudson commented on HDFS-9180: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2400 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2400/]) HDFS-9180. Update excluded DataNodes in DFSStripedOutputStream based on (jing9: rev a8b4d0ff283a0af1075aaa94904d4c6e63a9a3dd) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestWriteReadStripedFile.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripedDataStreamer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReadStripedFileWithDecoding.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/StripedFileTestUtil.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Fix For: 3.0.0 > > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch, HDFS-9180.003.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14945674#comment-14945674 ] Hudson commented on HDFS-9180: -- SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #495 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/495/]) HDFS-9180. Update excluded DataNodes in DFSStripedOutputStream based on (jing9: rev a8b4d0ff283a0af1075aaa94904d4c6e63a9a3dd) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReadStripedFileWithDecoding.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestWriteReadStripedFile.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripedDataStreamer.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/StripedFileTestUtil.java > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Fix For: 3.0.0 > > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch, HDFS-9180.003.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14945605#comment-14945605 ] Hudson commented on HDFS-9180: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2430 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2430/]) HDFS-9180. Update excluded DataNodes in DFSStripedOutputStream based on (jing9: rev a8b4d0ff283a0af1075aaa94904d4c6e63a9a3dd) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/StripedFileTestUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReadStripedFileWithDecoding.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestWriteReadStripedFile.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripedDataStreamer.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStream.java > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Fix For: 3.0.0 > > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch, HDFS-9180.003.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14945538#comment-14945538 ] Hudson commented on HDFS-9180: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #486 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/486/]) HDFS-9180. Update excluded DataNodes in DFSStripedOutputStream based on (jing9: rev a8b4d0ff283a0af1075aaa94904d4c6e63a9a3dd) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestWriteReadStripedFile.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReadStripedFileWithDecoding.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/StripedFileTestUtil.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripedDataStreamer.java > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Fix For: 3.0.0 > > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch, HDFS-9180.003.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14945479#comment-14945479 ] Hudson commented on HDFS-9180: -- FAILURE: Integrated in Hadoop-trunk-Commit #8576 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8576/]) HDFS-9180. Update excluded DataNodes in DFSStripedOutputStream based on (jing9: rev a8b4d0ff283a0af1075aaa94904d4c6e63a9a3dd) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/StripedFileTestUtil.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestWriteReadStripedFile.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReadStripedFileWithDecoding.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripedDataStreamer.java > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Fix For: 3.0.0 > > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch, HDFS-9180.003.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14944873#comment-14944873 ] Yi Liu commented on HDFS-9180: -- +1, thanks [~jingzhao], also thanks Walter for the review. > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch, HDFS-9180.003.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14944529#comment-14944529 ] Jing Zhao commented on HDFS-9180: - Thanks Walter! [~hitliuyi], could you please take a look at the 03 patch again? Thanks! > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch, HDFS-9180.003.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14944422#comment-14944422 ] Walter Su commented on HDFS-9180: - Thanks, Jing! The 03 patch LGTM. > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch, HDFS-9180.003.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14944371#comment-14944371 ] Hadoop QA commented on HDFS-9180: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 19m 45s | Pre-patch trunk has 7 extant Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 5 new or modified test files. | | {color:green}+1{color} | javac | 7m 55s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 9s | There were no new javadoc warning messages. | | {color:red}-1{color} | release audit | 0m 16s | The applied patch generated 1 release audit warnings. | | {color:green}+1{color} | checkstyle | 2m 57s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 28s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 4m 32s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 11s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 171m 10s | Tests failed in hadoop-hdfs. | | {color:green}+1{color} | hdfs tests | 0m 35s | Tests passed in hadoop-hdfs-client. | | | | 222m 37s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.hdfs.server.namenode.TestFSEditLogLoader | | | hadoop.hdfs.server.datanode.TestDirectoryScanner | | | hadoop.hdfs.web.TestWebHdfsTimeouts | | | hadoop.hdfs.security.TestDelegationTokenForProxyUser | | | hadoop.hdfs.web.TestWebHDFSOAuth2 | | | hadoop.hdfs.web.TestWebHdfsContentLength | | Timed out tests | org.apache.hadoop.hdfs.TestDFSClientRetries | | | org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshot | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12765069/HDFS-9180.003.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / b59e434 | | Pre-patch Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12799/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs-client.html | | Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12799/artifact/patchprocess/patchReleaseAuditProblems.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12799/artifact/patchprocess/testrun_hadoop-hdfs.txt | | hadoop-hdfs-client test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12799/artifact/patchprocess/testrun_hadoop-hdfs-client.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12799/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12799/console | This message was automatically generated. > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch, HDFS-9180.003.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14942662#comment-14942662 ] Walter Su commented on HDFS-9180: - 1. After calling replaceFailedStreamers(), does getExcludedNodes() always return empty? // allocateNewBlock() bq. When replacing a StripedDataStreamer, should also reset the currentPacket to null if the streamer is the first one. 2. You assumed replaceFailedStreamers() will call when currentIdx==0. That's true. It's just I think if we change {code} - if (i == 0) { + if (i == getCurrentIndex()) { {code} it's more readable. And getCurrentIndex() can get index from {{currentBlockGroup}} instead of {{streamer}}. 3. I don't know why #7 streamer didn't bump gs in the failed test. I run 40min and can't reproduce. If we check blockGroup.numBytes() instead of DATA_STREAMING, and assert stage is DATA_STREAMING, does it help debug if test fails again? {code} if (streamer.isHealthy() && // the streamer may not be in STREAMING stage if the block length is // less than a stripe streamer.getStage() == BlockConstructionStage.DATA_STREAMING) { streamer.setExternalError(); healthySet.add(streamer); } {code} #2, #3 not related. > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14942095#comment-14942095 ] Hadoop QA commented on HDFS-9180: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 20m 13s | Pre-patch trunk has 7 extant Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 4 new or modified test files. | | {color:green}+1{color} | javac | 8m 9s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 17s | There were no new javadoc warning messages. | | {color:red}-1{color} | release audit | 0m 16s | The applied patch generated 1 release audit warnings. | | {color:green}+1{color} | checkstyle | 3m 1s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 32s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 4m 34s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 16s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 228m 58s | Tests failed in hadoop-hdfs. | | {color:green}+1{color} | hdfs tests | 0m 31s | Tests passed in hadoop-hdfs-client. | | | | 281m 28s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure | | | hadoop.hdfs.TestRollingUpgrade | | | hadoop.hdfs.TestLeaseRecovery2 | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12764816/HDFS-9180.002.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / fdf02d1 | | Pre-patch Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12780/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs-client.html | | Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12780/artifact/patchprocess/patchReleaseAuditProblems.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12780/artifact/patchprocess/testrun_hadoop-hdfs.txt | | hadoop-hdfs-client test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12780/artifact/patchprocess/testrun_hadoop-hdfs-client.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12780/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12780/console | This message was automatically generated. > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14942000#comment-14942000 ] Yi Liu commented on HDFS-9180: -- {quote} I kicked the Jenkins again. If we all agree the fixes here are valid maybe we can consider commit them first and keep fixing remaining issues in separate jiras. {quote} +1, thanks Jing. > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941986#comment-14941986 ] Jing Zhao commented on HDFS-9180: - {{TestDFSStripedOutputStreamWithFailure#testDatanodeFailureRandomLength}} failed with length {{1835007}}. I've run with the same length locally 150 times but could not reproduce the failure. I kicked the Jenkins again. If we all agree the fixes here are valid maybe we can consider commit them first and keep fixing remaining issues in separate jiras. > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941913#comment-14941913 ] Hadoop QA commented on HDFS-9180: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 19m 39s | Pre-patch trunk has 7 extant Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 4 new or modified test files. | | {color:green}+1{color} | javac | 7m 52s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 2s | There were no new javadoc warning messages. | | {color:red}-1{color} | release audit | 0m 16s | The applied patch generated 1 release audit warnings. | | {color:green}+1{color} | checkstyle | 2m 52s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 27s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 4m 28s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 11s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 188m 46s | Tests failed in hadoop-hdfs. | | {color:green}+1{color} | hdfs tests | 0m 32s | Tests passed in hadoop-hdfs-client. | | | | 239m 43s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.hdfs.web.TestWebHDFSOAuth2 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure | | | hadoop.hdfs.TestParallelShortCircuitRead | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12764816/HDFS-9180.002.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / a68b6eb | | Pre-patch Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12775/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs-client.html | | Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12775/artifact/patchprocess/patchReleaseAuditProblems.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12775/artifact/patchprocess/testrun_hadoop-hdfs.txt | | hadoop-hdfs-client test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12775/artifact/patchprocess/testrun_hadoop-hdfs-client.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12775/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12775/console | This message was automatically generated. > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch, > HDFS-9180.002.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941035#comment-14941035 ] Hadoop QA commented on HDFS-9180: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 21m 0s | Findbugs (version ) appears to be broken on trunk. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 3 new or modified test files. | | {color:green}+1{color} | javac | 9m 1s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 11m 13s | There were no new javadoc warning messages. | | {color:red}-1{color} | release audit | 0m 15s | The applied patch generated 1 release audit warnings. | | {color:green}+1{color} | checkstyle | 2m 11s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 40s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 37s | The patch built with eclipse:eclipse. | | {color:red}-1{color} | findbugs | 5m 5s | The patch appears to introduce 7 new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 40s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 239m 52s | Tests failed in hadoop-hdfs. | | {color:green}+1{color} | hdfs tests | 0m 33s | Tests passed in hadoop-hdfs-client. | | | | 295m 12s | | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-hdfs-client | | Failed unit tests | hadoop.hdfs.TestRecoverStripedFile | | | hadoop.hdfs.TestRollingUpgrade | | | hadoop.hdfs.TestWriteReadStripedFile | | | hadoop.hdfs.TestDFSStripedOutputStream | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12764700/HDFS-9180.001.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / fd026f5 | | Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12771/artifact/patchprocess/patchReleaseAuditProblems.txt | | Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12771/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs-client.html | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12771/artifact/patchprocess/testrun_hadoop-hdfs.txt | | hadoop-hdfs-client test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12771/artifact/patchprocess/testrun_hadoop-hdfs-client.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12771/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12771/console | This message was automatically generated. > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940679#comment-14940679 ] Yi Liu commented on HDFS-9180: -- Thanks [~jingzhao] for the work. I have spent some time to familiar with what you guys did in HDFS-9040 :). The patch looks good to me. +1 pending new Jenkins. For the test, maybe we can add it and just comment out {{@Test}}. > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940677#comment-14940677 ] Hadoop QA commented on HDFS-9180: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 19m 39s | Pre-patch trunk has 7 extant Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 3 new or modified test files. | | {color:green}+1{color} | javac | 7m 45s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 1s | There were no new javadoc warning messages. | | {color:red}-1{color} | release audit | 0m 16s | The applied patch generated 1 release audit warnings. | | {color:green}+1{color} | checkstyle | 2m 56s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 30s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 4m 31s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 24s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 102m 5s | Tests failed in hadoop-hdfs. | | {color:green}+1{color} | hdfs tests | 0m 36s | Tests passed in hadoop-hdfs-client. | | | | 153m 24s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits | | | hadoop.hdfs.server.mover.TestMover | | | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA | | | hadoop.hdfs.server.namenode.ha.TestHAFsck | | | hadoop.hdfs.server.namenode.TestCreateEditsLog | | | hadoop.hdfs.server.namenode.TestNamenodeRetryCache | | | hadoop.hdfs.server.namenode.ha.TestNNHealthCheck | | | hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement | | | hadoop.hdfs.server.namenode.ha.TestStandbyIsHot | | | hadoop.hdfs.server.namenode.ha.TestStateTransitionFailure | | | hadoop.hdfs.server.namenode.ha.TestEditLogTailer | | | hadoop.hdfs.server.namenode.TestBackupNode | | | hadoop.hdfs.server.namenode.ha.TestHAStateTransitions | | | hadoop.hdfs.server.namenode.ha.TestQuotasWithHA | | | hadoop.hdfs.server.namenode.ha.TestHAAppend | | | hadoop.hdfs.server.namenode.ha.TestHAMetrics | | | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover | | | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA | | | hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade | | | hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics | | | hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover | | Timed out tests | org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport | | | org.apache.hadoop.hdfs.server.namenode.TestFileTruncate | | | org.apache.hadoop.hdfs.server.mover.TestStorageMover | | | org.apache.hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir | | | org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12764700/HDFS-9180.001.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / fd026f5 | | Pre-patch Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12766/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs-client.html | | Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12766/artifact/patchprocess/patchReleaseAuditProblems.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12766/artifact/patchprocess/testrun_hadoop-hdfs.txt | | hadoop-hdfs-client test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12766/artifact/patchprocess/testrun_hadoop-hdfs-client.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12766/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12766/console | This message was automatically generated. > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940564#comment-14940564 ] Jing Zhao commented on HDFS-9180: - To help testing the patch, we can add the following test in {{TestDFSStripedOutputStreamWithFailure}}: {code} @Test public void testDataFailureWithAllLengths() throws Exception { for (int length : LENGTHS) { LOG.info("run test with length: " + length); runTest(length); } } {code} The test may take a couple of hours. > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Attachments: HDFS-9180.000.patch, HDFS-9180.001.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940042#comment-14940042 ] Walter Su commented on HDFS-9180: - Thanks Jing for the great work. I'm on holiday. I'll take a look tomorrow. Maybe others can help review together. > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Attachments: HDFS-9180.000.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939358#comment-14939358 ] Hadoop QA commented on HDFS-9180: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 20m 5s | Pre-patch trunk has 7 extant Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 3 new or modified test files. | | {color:green}+1{color} | javac | 7m 48s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 27s | There were no new javadoc warning messages. | | {color:red}-1{color} | release audit | 0m 16s | The applied patch generated 1 release audit warnings. | | {color:green}+1{color} | checkstyle | 3m 5s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 36s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 38s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 5m 40s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 14s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 193m 18s | Tests failed in hadoop-hdfs. | | {color:green}+1{color} | hdfs tests | 0m 33s | Tests passed in hadoop-hdfs-client. | | | | 246m 46s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner | | | hadoop.hdfs.server.blockmanagement.TestBlockManager | | | hadoop.hdfs.web.TestWebHDFSOAuth2 | | | hadoop.hdfs.TestRecoverStripedFile | | | hadoop.hdfs.server.namenode.TestFSNamesystem | | Timed out tests | org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager | | | org.apache.hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes | | | org.apache.hadoop.hdfs.TestGetBlocks | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12764525/HDFS-9180.000.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 5db371f | | Pre-patch Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12757/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs-client.html | | Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12757/artifact/patchprocess/patchReleaseAuditProblems.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12757/artifact/patchprocess/testrun_hadoop-hdfs.txt | | hadoop-hdfs-client test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12757/artifact/patchprocess/testrun_hadoop-hdfs-client.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12757/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12757/console | This message was automatically generated. > Update excluded DataNodes in DFSStripedOutputStream based on failures in data > streamers > --- > > Key: HDFS-9180 > URL: https://issues.apache.org/jira/browse/HDFS-9180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Affects Versions: 3.0.0 >Reporter: Jing Zhao >Assignee: Jing Zhao > Attachments: HDFS-9180.000.patch > > > This is a TODO in HDFS-9040: based on the failures all the striped data > streamers hit, the DFSStripedOutputStream should keep a record of all the > DataNodes that should be excluded. > This jira will also fix several bugs in the DFSStripedOutputStream. Will > provide more details in the comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)