[jira] [Commented] (HDFS-8282) Erasure coding: move striped reading logic to StripedBlockUtil
[ https://issues.apache.org/jira/browse/HDFS-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938549#comment-14938549 ] Hudson commented on HDFS-8282: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #465 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/465/]) HDFS-8282. Erasure coding: move striped reading logic to (zhezhang: rev f0628280c351640a807b05eaf786d051b5a7bd66) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestPlanReadPortions.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedInputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/StripedBlockUtil.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt > Erasure coding: move striped reading logic to StripedBlockUtil > -- > > Key: HDFS-8282 > URL: https://issues.apache.org/jira/browse/HDFS-8282 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Zhe Zhang >Assignee: Zhe Zhang > Fix For: HDFS-7285 > > Attachments: HDFS-8282-HDFS-7285.00.patch, > HDFS-8282-HDFS-7285.01.patch, HDFS-8282-HDFS-7285.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8282) Erasure coding: move striped reading logic to StripedBlockUtil
[ https://issues.apache.org/jira/browse/HDFS-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938354#comment-14938354 ] Hudson commented on HDFS-8282: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2408 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2408/]) HDFS-8282. Erasure coding: move striped reading logic to (zhezhang: rev f0628280c351640a807b05eaf786d051b5a7bd66) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/StripedBlockUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestPlanReadPortions.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedInputStream.java > Erasure coding: move striped reading logic to StripedBlockUtil > -- > > Key: HDFS-8282 > URL: https://issues.apache.org/jira/browse/HDFS-8282 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Zhe Zhang >Assignee: Zhe Zhang > Fix For: HDFS-7285 > > Attachments: HDFS-8282-HDFS-7285.00.patch, > HDFS-8282-HDFS-7285.01.patch, HDFS-8282-HDFS-7285.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8282) Erasure coding: move striped reading logic to StripedBlockUtil
[ https://issues.apache.org/jira/browse/HDFS-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937979#comment-14937979 ] Hudson commented on HDFS-8282: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #439 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/439/]) HDFS-8282. Erasure coding: move striped reading logic to (zhezhang: rev f0628280c351640a807b05eaf786d051b5a7bd66) * hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestPlanReadPortions.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/StripedBlockUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedInputStream.java > Erasure coding: move striped reading logic to StripedBlockUtil > -- > > Key: HDFS-8282 > URL: https://issues.apache.org/jira/browse/HDFS-8282 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Zhe Zhang >Assignee: Zhe Zhang > Fix For: HDFS-7285 > > Attachments: HDFS-8282-HDFS-7285.00.patch, > HDFS-8282-HDFS-7285.01.patch, HDFS-8282-HDFS-7285.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8282) Erasure coding: move striped reading logic to StripedBlockUtil
[ https://issues.apache.org/jira/browse/HDFS-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937759#comment-14937759 ] Hudson commented on HDFS-8282: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #473 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/473/]) HDFS-8282. Erasure coding: move striped reading logic to (zhezhang: rev f0628280c351640a807b05eaf786d051b5a7bd66) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestPlanReadPortions.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/StripedBlockUtil.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedInputStream.java > Erasure coding: move striped reading logic to StripedBlockUtil > -- > > Key: HDFS-8282 > URL: https://issues.apache.org/jira/browse/HDFS-8282 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Zhe Zhang >Assignee: Zhe Zhang > Fix For: HDFS-7285 > > Attachments: HDFS-8282-HDFS-7285.00.patch, > HDFS-8282-HDFS-7285.01.patch, HDFS-8282-HDFS-7285.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8282) Erasure coding: move striped reading logic to StripedBlockUtil
[ https://issues.apache.org/jira/browse/HDFS-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937503#comment-14937503 ] Hudson commented on HDFS-8282: -- FAILURE: Integrated in Hadoop-Yarn-trunk #1203 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1203/]) HDFS-8282. Erasure coding: move striped reading logic to (zhezhang: rev f0628280c351640a807b05eaf786d051b5a7bd66) * hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestPlanReadPortions.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/StripedBlockUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedInputStream.java > Erasure coding: move striped reading logic to StripedBlockUtil > -- > > Key: HDFS-8282 > URL: https://issues.apache.org/jira/browse/HDFS-8282 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Zhe Zhang >Assignee: Zhe Zhang > Fix For: HDFS-7285 > > Attachments: HDFS-8282-HDFS-7285.00.patch, > HDFS-8282-HDFS-7285.01.patch, HDFS-8282-HDFS-7285.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8282) Erasure coding: move striped reading logic to StripedBlockUtil
[ https://issues.apache.org/jira/browse/HDFS-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937253#comment-14937253 ] Hudson commented on HDFS-8282: -- FAILURE: Integrated in Hadoop-trunk-Commit #8548 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8548/]) HDFS-8282. Erasure coding: move striped reading logic to (zhezhang: rev f0628280c351640a807b05eaf786d051b5a7bd66) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedInputStream.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestPlanReadPortions.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/StripedBlockUtil.java > Erasure coding: move striped reading logic to StripedBlockUtil > -- > > Key: HDFS-8282 > URL: https://issues.apache.org/jira/browse/HDFS-8282 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Zhe Zhang >Assignee: Zhe Zhang > Fix For: HDFS-7285 > > Attachments: HDFS-8282-HDFS-7285.00.patch, > HDFS-8282-HDFS-7285.01.patch, HDFS-8282-HDFS-7285.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8282) Erasure coding: move striped reading logic to StripedBlockUtil
[ https://issues.apache.org/jira/browse/HDFS-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937057#comment-14937057 ] Hudson commented on HDFS-8282: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2379 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2379/]) HDFS-8282. Erasure coding: move striped reading logic to (zhezhang: rev f0628280c351640a807b05eaf786d051b5a7bd66) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedInputStream.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/StripedBlockUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestPlanReadPortions.java > Erasure coding: move striped reading logic to StripedBlockUtil > -- > > Key: HDFS-8282 > URL: https://issues.apache.org/jira/browse/HDFS-8282 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Zhe Zhang >Assignee: Zhe Zhang > Fix For: HDFS-7285 > > Attachments: HDFS-8282-HDFS-7285.00.patch, > HDFS-8282-HDFS-7285.01.patch, HDFS-8282-HDFS-7285.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8282) Erasure coding: move striped reading logic to StripedBlockUtil
[ https://issues.apache.org/jira/browse/HDFS-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520994#comment-14520994 ] Zhe Zhang commented on HDFS-8282: - Thanks Yi for reviewing again! I just committed it to the branch. > Erasure coding: move striped reading logic to StripedBlockUtil > -- > > Key: HDFS-8282 > URL: https://issues.apache.org/jira/browse/HDFS-8282 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Zhe Zhang >Assignee: Zhe Zhang > Attachments: HDFS-8282-HDFS-7285.00.patch, > HDFS-8282-HDFS-7285.01.patch, HDFS-8282-HDFS-7285.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8282) Erasure coding: move striped reading logic to StripedBlockUtil
[ https://issues.apache.org/jira/browse/HDFS-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520989#comment-14520989 ] Yi Liu commented on HDFS-8282: -- yes, +1 > Erasure coding: move striped reading logic to StripedBlockUtil > -- > > Key: HDFS-8282 > URL: https://issues.apache.org/jira/browse/HDFS-8282 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Zhe Zhang >Assignee: Zhe Zhang > Attachments: HDFS-8282-HDFS-7285.00.patch, > HDFS-8282-HDFS-7285.01.patch, HDFS-8282-HDFS-7285.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8282) Erasure coding: move striped reading logic to StripedBlockUtil
[ https://issues.apache.org/jira/browse/HDFS-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520937#comment-14520937 ] Hadoop QA commented on HDFS-8282: - (!) The patch artifact directory on has been removed! This is a fatal error for test-patch.sh. Aborting. Jenkins (node H4) information at https://builds.apache.org/job/PreCommit-HDFS-Build/10469/ may provide some hints. > Erasure coding: move striped reading logic to StripedBlockUtil > -- > > Key: HDFS-8282 > URL: https://issues.apache.org/jira/browse/HDFS-8282 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Zhe Zhang >Assignee: Zhe Zhang > Attachments: HDFS-8282-HDFS-7285.00.patch, > HDFS-8282-HDFS-7285.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8282) Erasure coding: move striped reading logic to StripedBlockUtil
[
https://issues.apache.org/jira/browse/HDFS-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520683#comment-14520683
]
Hadoop QA commented on HDFS-8282:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 15m 5s | Pre-patch HDFS-7285 compilation
is healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:red}-1{color} | javac | 7m 46s | The applied patch generated 8
additional warning messages. |
| {color:green}+1{color} | javadoc | 9m 56s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 15s | The applied patch generated
1 release audit warnings. |
| {color:red}-1{color} | checkstyle | 4m 4s | The applied patch generated 4
additional checkstyle issues. |
| {color:green}+1{color} | install | 1m 38s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 17s | The patch appears to introduce
10 new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | native | 3m 21s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 62m 0s | Tests failed in hadoop-hdfs. |
| | | 108m 2s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| | Inconsistent synchronization of
org.apache.hadoop.hdfs.DFSOutputStream.streamer; locked 90% of time
Unsynchronized access at DFSOutputStream.java:90% of time Unsynchronized
access at DFSOutputStream.java:[line 142] |
| | Switch statement found in
org.apache.hadoop.hdfs.DFSStripedInputStream.fetchBlockByteRange(long, long,
long, byte[], int, Map) where default case is missing At
DFSStripedInputStream.java:long, long, byte[], int, Map) where default case is
missing At DFSStripedInputStream.java:[lines 468-491] |
| | Class org.apache.hadoop.hdfs.DataStreamer$LastException is not derived
from an Exception, even though it is named as such At DataStreamer.java:from
an Exception, even though it is named as such At DataStreamer.java:[lines
177-201] |
| | Dead store to offSuccess in
org.apache.hadoop.hdfs.StripedDataStreamer.endBlock() At
StripedDataStreamer.java:org.apache.hadoop.hdfs.StripedDataStreamer.endBlock()
At StripedDataStreamer.java:[line 105] |
| | Result of integer multiplication cast to long in
org.apache.hadoop.hdfs.server.blockmanagement.BlockInfoStriped.spaceConsumed()
At BlockInfoStriped.java:to long in
org.apache.hadoop.hdfs.server.blockmanagement.BlockInfoStriped.spaceConsumed()
At BlockInfoStriped.java:[line 208] |
| | Possible null pointer dereference of arr$ in
org.apache.hadoop.hdfs.server.blockmanagement.BlockInfoStripedUnderConstruction.initializeBlockRecovery(long)
Dereferenced at BlockInfoStripedUnderConstruction.java:arr$ in
org.apache.hadoop.hdfs.server.blockmanagement.BlockInfoStripedUnderConstruction.initializeBlockRecovery(long)
Dereferenced at BlockInfoStripedUnderConstruction.java:[line 206] |
| | Found reliance on default encoding in
org.apache.hadoop.hdfs.server.namenode.ErasureCodingZoneManager.createErasureCodingZone(String,
ECSchema):in
org.apache.hadoop.hdfs.server.namenode.ErasureCodingZoneManager.createErasureCodingZone(String,
ECSchema): String.getBytes() At ErasureCodingZoneManager.java:[line 116] |
| | Found reliance on default encoding in
org.apache.hadoop.hdfs.server.namenode.ErasureCodingZoneManager.getECZoneInfo(INodesInPath):in
org.apache.hadoop.hdfs.server.namenode.ErasureCodingZoneManager.getECZoneInfo(INodesInPath):
new String(byte[]) At ErasureCodingZoneManager.java:[line 81] |
| | Result of integer multiplication cast to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.constructInternalBlock(LocatedStripedBlock,
int, int, int, int) At StripedBlockUtil.java:to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.constructInternalBlock(LocatedStripedBlock,
int, int, int, int) At StripedBlockUtil.java:[line 85] |
| | Result of integer multiplication cast to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.planReadPortions(int, int, long,
int, int) At StripedBlockUtil.java:to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.planReadPortions(int, int, long,
int, int) At StripedBlockUtil.java:[line 167] |
| Failed unit tests | hadoop.hdfs.TestQuota |
| | hadoop.fs.TestUrlStreamHandler |
| | hadoop.hdfs.TestClose |
| Timed out tests | org.apache.hadoop.hdfs.TestData
[jira] [Commented] (HDFS-8282) Erasure coding: move striped reading logic to StripedBlockUtil
[ https://issues.apache.org/jira/browse/HDFS-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520640#comment-14520640 ] Yi Liu commented on HDFS-8282: -- +1, thanks Zhe. > Erasure coding: move striped reading logic to StripedBlockUtil > -- > > Key: HDFS-8282 > URL: https://issues.apache.org/jira/browse/HDFS-8282 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Zhe Zhang >Assignee: Zhe Zhang > Attachments: HDFS-8282-HDFS-7285.00.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
