[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14937840#comment-14937840 ] Hudson commented on HADOOP-11920: - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #473 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/473/]) HADOOP-11920. Refactor some codes for erasure coders. Contributed by Kai (zhezhang: rev 09c3a375bafa481e88d1317388a73c46950164c9) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/XORRawEncoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RawErasureDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RawErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/XORRawDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/ECChunk.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/AbstractErasureCoder.java * hadoop-common-project/hadoop-common/CHANGES-HDFS-EC-7285.txt * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/TestCoderBase.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureEncoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/RSErasureDecoder.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/coder/TestRSErasureCoder.java > Refactor some codes for erasure coders > -- > > Key: HADOOP-11920 > URL: https://issues.apache.org/jira/browse/HADOOP-11920 > Project: Hadoop Common > Issue Type: Sub-task > Components: io >Reporter: Kai Zheng >Assignee: Kai Zheng > Fix For: HDFS-7285 > > Attachments: HADOOP-11920-HDFS-7285-02.patch, > HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-HDFS-7285-v5.patch, > HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch > > > While working on native erasure coders and also HADOOP-11847, it was found in > some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938059#comment-14938059 ] Hudson commented on HADOOP-11920: - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #439 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/439/]) HADOOP-11920. Refactor some codes for erasure coders. Contributed by Kai (zhezhang: rev 09c3a375bafa481e88d1317388a73c46950164c9) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureDecoder.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/coder/TestRSErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/AbstractErasureCoder.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/TestCoderBase.java * hadoop-common-project/hadoop-common/CHANGES-HDFS-EC-7285.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RawErasureDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/XORRawEncoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/ECChunk.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureEncoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/RSErasureDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/XORRawDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RawErasureCoder.java > Refactor some codes for erasure coders > -- > > Key: HADOOP-11920 > URL: https://issues.apache.org/jira/browse/HADOOP-11920 > Project: Hadoop Common > Issue Type: Sub-task > Components: io >Reporter: Kai Zheng >Assignee: Kai Zheng > Fix For: HDFS-7285 > > Attachments: HADOOP-11920-HDFS-7285-02.patch, > HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-HDFS-7285-v5.patch, > HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch > > > While working on native erasure coders and also HADOOP-11847, it was found in > some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938437#comment-14938437 ] Hudson commented on HADOOP-11920: - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2408 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2408/]) HADOOP-11920. Refactor some codes for erasure coders. Contributed by Kai (zhezhang: rev 09c3a375bafa481e88d1317388a73c46950164c9) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RawErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/XORRawEncoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/AbstractErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/XORRawDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureEncoder.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/TestCoderBase.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/RSErasureDecoder.java * hadoop-common-project/hadoop-common/CHANGES-HDFS-EC-7285.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/ECChunk.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureCoder.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/coder/TestRSErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RawErasureDecoder.java > Refactor some codes for erasure coders > -- > > Key: HADOOP-11920 > URL: https://issues.apache.org/jira/browse/HADOOP-11920 > Project: Hadoop Common > Issue Type: Sub-task > Components: io >Reporter: Kai Zheng >Assignee: Kai Zheng > Fix For: HDFS-7285 > > Attachments: HADOOP-11920-HDFS-7285-02.patch, > HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-HDFS-7285-v5.patch, > HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch > > > While working on native erasure coders and also HADOOP-11847, it was found in > some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938629#comment-14938629 ] Hudson commented on HADOOP-11920: - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #465 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/465/]) HADOOP-11920. Refactor some codes for erasure coders. Contributed by Kai (zhezhang: rev 09c3a375bafa481e88d1317388a73c46950164c9) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/AbstractErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/XORRawEncoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/XORRawDecoder.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/TestCoderBase.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/ECChunk.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureEncoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/RSErasureDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RawErasureDecoder.java * hadoop-common-project/hadoop-common/CHANGES-HDFS-EC-7285.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureDecoder.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/coder/TestRSErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RawErasureCoder.java > Refactor some codes for erasure coders > -- > > Key: HADOOP-11920 > URL: https://issues.apache.org/jira/browse/HADOOP-11920 > Project: Hadoop Common > Issue Type: Sub-task > Components: io >Reporter: Kai Zheng >Assignee: Kai Zheng > Fix For: HDFS-7285 > > Attachments: HADOOP-11920-HDFS-7285-02.patch, > HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-HDFS-7285-v5.patch, > HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch > > > While working on native erasure coders and also HADOOP-11847, it was found in > some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14937583#comment-14937583 ] Hudson commented on HADOOP-11920: - FAILURE: Integrated in Hadoop-Yarn-trunk #1203 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1203/]) HADOOP-11920. Refactor some codes for erasure coders. Contributed by Kai (zhezhang: rev 09c3a375bafa481e88d1317388a73c46950164c9) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureEncoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/XORRawDecoder.java * hadoop-common-project/hadoop-common/CHANGES-HDFS-EC-7285.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RawErasureCoder.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/TestCoderBase.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/coder/TestRSErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/ECChunk.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/XORRawEncoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/RSErasureDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RawErasureDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/AbstractErasureCoder.java > Refactor some codes for erasure coders > -- > > Key: HADOOP-11920 > URL: https://issues.apache.org/jira/browse/HADOOP-11920 > Project: Hadoop Common > Issue Type: Sub-task > Components: io >Reporter: Kai Zheng >Assignee: Kai Zheng > Fix For: HDFS-7285 > > Attachments: HADOOP-11920-HDFS-7285-02.patch, > HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-HDFS-7285-v5.patch, > HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch > > > While working on native erasure coders and also HADOOP-11847, it was found in > some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14937336#comment-14937336 ] Hudson commented on HADOOP-11920: - FAILURE: Integrated in Hadoop-trunk-Commit #8548 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8548/]) HADOOP-11920. Refactor some codes for erasure coders. Contributed by Kai (zhezhang: rev 09c3a375bafa481e88d1317388a73c46950164c9) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/ECChunk.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/RSErasureDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/XORRawDecoder.java * hadoop-common-project/hadoop-common/CHANGES-HDFS-EC-7285.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RawErasureCoder.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/TestCoderBase.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/XORRawEncoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/AbstractErasureCoder.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/coder/TestRSErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RawErasureDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureEncoder.java > Refactor some codes for erasure coders > -- > > Key: HADOOP-11920 > URL: https://issues.apache.org/jira/browse/HADOOP-11920 > Project: Hadoop Common > Issue Type: Sub-task > Components: io >Reporter: Kai Zheng >Assignee: Kai Zheng > Fix For: HDFS-7285 > > Attachments: HADOOP-11920-HDFS-7285-02.patch, > HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-HDFS-7285-v5.patch, > HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch > > > While working on native erasure coders and also HADOOP-11847, it was found in > some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14937139#comment-14937139 ] Hudson commented on HADOOP-11920: - FAILURE: Integrated in Hadoop-Hdfs-trunk #2379 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2379/]) HADOOP-11920. Refactor some codes for erasure coders. Contributed by Kai (zhezhang: rev 09c3a375bafa481e88d1317388a73c46950164c9) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/XORRawDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureEncoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/RSErasureDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureDecoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RawErasureCoder.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/coder/TestRSErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/ECChunk.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RawErasureDecoder.java * hadoop-common-project/hadoop-common/CHANGES-HDFS-EC-7285.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/AbstractErasureCoder.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/AbstractRawErasureCoder.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/TestCoderBase.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/XORRawEncoder.java > Refactor some codes for erasure coders > -- > > Key: HADOOP-11920 > URL: https://issues.apache.org/jira/browse/HADOOP-11920 > Project: Hadoop Common > Issue Type: Sub-task > Components: io >Reporter: Kai Zheng >Assignee: Kai Zheng > Fix For: HDFS-7285 > > Attachments: HADOOP-11920-HDFS-7285-02.patch, > HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-HDFS-7285-v5.patch, > HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch > > > While working on native erasure coders and also HADOOP-11847, it was found in > some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544485#comment-14544485 ] Zhe Zhang commented on HADOOP-11920: Since there's only HADOOP-11566 ahead of these 2, shall we revert them and re-commit? If it sounds good to you I can take care of it. Please just attach the correct patches to each JIRA and let me know the order of the 2 to commit. --- Zhe Zhang Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Fix For: HDFS-7285 Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-HDFS-7285-v5.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544632#comment-14544632 ] Kai Zheng commented on HADOOP-11920: If that will result in cleaner commit logs, I'm OK. The two patches were correctly attached on the two issues respectively, and we should commit HADOOP-11921 first. Thanks for the taking! Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Fix For: HDFS-7285 Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-HDFS-7285-v5.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538868#comment-14538868 ] Kai Zheng commented on HADOOP-11920: Commits of HADOOP-11920 and HADOOP-11921 are kinds of messy and confusing even to me now. If the combining will keep the two entries, I guess it's fine. Thanks. Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Fix For: HDFS-7285 Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-HDFS-7285-v5.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538183#comment-14538183 ] Zhe Zhang commented on HADOOP-11920: {code} 101 pick 3140bc0 HADOOP-11921. Enhance tests for erasure coders. Contributed by Kai Zheng 102 pick 88e6c42 HADOOP-11921 Correct the JIRA entry in CHANGES-HDFS-EC-7285.txt 103 pick ca32b8f HDFS-8334. Erasure coding: rename DFSStripedInputStream related test classes. Contributed by Zhe Zhang. 104 pick 16ba1a5 Revert mistakenly committing of HADOOP-11920 patch 105 pick 0f7eb46 HADOOP-11921. Enhance tests for erasure coders. Contributed by Kai Zheng 106 pick 10bfa92 HDFS-8129. Erasure Coding: Maintain consistent naming for Erasure Coding related classes - EC/ErasureCoding. Contributed by Uma Maheswara Rao G 107 pick 2a89e1d HADOOP-11920. Refactor some codes for erasure coders. Contributed by Kai Zheng {code} In the next rebase I'd like to cleanup the #104 commit. Should I combine it with 101/102? Thanks. Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Fix For: HDFS-7285 Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-HDFS-7285-v5.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14530146#comment-14530146 ] Vinayakumar B commented on HADOOP-11920: Latest patch looks good. One small nit 1. In RawErasureDecoder javadoc should be updated for {{decode(ECChunk\[\] inputs, int\[\] erasedIndexes, ECChunk\[\] outputs)}} also. Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14530177#comment-14530177 ] Kai Zheng commented on HADOOP-11920: Thanks Vinay for the review and help. I will wait for the Jenkins building output to update the patch. Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14530270#comment-14530270 ] Hadoop QA commented on HADOOP-11920: \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 14m 41s | Pre-patch HDFS-7285 compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 2 new or modified test files. | | {color:green}+1{color} | javac | 7m 31s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 40s | There were no new javadoc warning messages. | | {color:red}-1{color} | release audit | 1m 8s | The applied patch generated 15 release audit warnings. | | {color:red}-1{color} | checkstyle | 1m 3s | The applied patch generated 5 new checkstyle issues (total was 112, now 112). | | {color:green}+1{color} | whitespace | 0m 2s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 33s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 40s | The patch does not introduce any new Findbugs (version 2.0.3) warnings. | | {color:green}+1{color} | common tests | 23m 43s | Tests passed in hadoop-common. | | | | 61m 38s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12730778/HADOOP-11920-HDFS-7285-02.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | HDFS-7285 / 850d7fa | | Release Audit | https://builds.apache.org/job/PreCommit-HADOOP-Build/6503/artifact/patchprocess/patchReleaseAuditProblems.txt | | checkstyle | https://builds.apache.org/job/PreCommit-HADOOP-Build/6503/artifact/patchprocess/diffcheckstylehadoop-common.txt | | hadoop-common test log | https://builds.apache.org/job/PreCommit-HADOOP-Build/6503/artifact/patchprocess/testrun_hadoop-common.txt | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/6503/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/6503/console | This message was automatically generated. Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531667#comment-14531667 ] Hadoop QA commented on HADOOP-11920: \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | patch | 0m 0s | The patch command could not apply the patch during dryrun. | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12730995/HADOOP-11920-HDFS-7285-v4.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | HDFS-7285 / 88e6c42 | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/6510/console | This message was automatically generated. Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531892#comment-14531892 ] Yi Liu commented on HADOOP-11920: - Yes, the caller can pass a direct buffer, but also can use a java heap byte buffer. Why it should be DirectByteBuffer? In RawErasureEncoder, the encode declares it accepts {{ByteBuffer}} {code} public void encode(ByteBuffer[] inputs, ByteBuffer[] outputs); {code} If we want to accept only Direct ByteBuffer in {{XORRawEncoder#doEncode}}, we should check it must be a direct buffer. Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531887#comment-14531887 ] Kai Zheng commented on HADOOP-11920: Hi Yi, Thanks for your comments. The buffer should be DirectByteBuffer. Raw erasure coders support two types of buffers: bytes array buffer and direct ByteBuffer. As you may know, native coders would love more direct ByteBuffers for better performance. In {{RawErasureCoder}}, it has: {code} /** * Tell if direct buffer is preferred or not. It's for callers to * decide how to allocate coding chunk buffers, using DirectByteBuffer or * bytes array. It will return false by default. * @return true if native buffer is preferred for performance consideration, * otherwise false. */ public boolean preferDirectBuffer(); {code} The Jenkins building failure was caused by my mistakenly committing of HADOOP-11921. I have re-committed the patch and will trigger new building soon. Yes it works for me to build it locally. Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531897#comment-14531897 ] Kai Zheng commented on HADOOP-11920: bq. Yes, the caller can pass a direct buffer, but also can use a java heap byte buffer. Why it should be DirectByteBuffer? I thought you're right. In that API callers should also certainly use heap buffers, though native coders prefer direct buffers. Let me update the patch allowing that with more double check. Thanks! Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531858#comment-14531858 ] Yi Liu commented on HADOOP-11920: - Thanks Kai for the patch. {quote} resetDirectBuffer(outputs[0]); {quote} The name should be resetBuffer, since it could be a java heap buffer? For the Jenkins, can you run locally for the related test case, if they pass, I think it's OK to go. Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14530714#comment-14530714 ] Hadoop QA commented on HADOOP-11920: \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | patch | 0m 0s | The patch command could not apply the patch during dryrun. | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12730847/HADOOP-11920-v3.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / a583a40 | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/6505/console | This message was automatically generated. Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531994#comment-14531994 ] Yi Liu commented on HADOOP-11920: - +1 , thanks Kai. Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-HDFS-7285-v5.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532003#comment-14532003 ] Kai Zheng commented on HADOOP-11920: Thanks Yi. I checked above Jenkins building output, the common test failure isn't relevant. I will address the minor unused import before the commit. Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-HDFS-7285-v5.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531975#comment-14531975 ] Hadoop QA commented on HADOOP-11920: \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 18m 11s | Pre-patch HDFS-7285 compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 2 new or modified test files. | | {color:green}+1{color} | javac | 9m 31s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 12m 26s | There were no new javadoc warning messages. | | {color:red}-1{color} | release audit | 1m 21s | The applied patch generated 12 release audit warnings. | | {color:red}-1{color} | checkstyle | 1m 22s | The applied patch generated 1 new checkstyle issues (total was 114, now 108). | | {color:green}+1{color} | whitespace | 0m 2s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 47s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 37s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 2m 3s | The patch does not introduce any new Findbugs (version 2.0.3) warnings. | | {color:red}-1{color} | common tests | 22m 49s | Tests failed in hadoop-common. | | | | 70m 25s | | \\ \\ || Reason || Tests || | Timed out tests | org.apache.hadoop.io.compress.TestCodec | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12731060/HADOOP-11920-HDFS-7285-v5.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | HDFS-7285 / 0f7eb46 | | Release Audit | https://builds.apache.org/job/PreCommit-HADOOP-Build/6511/artifact/patchprocess/patchReleaseAuditProblems.txt | | checkstyle | https://builds.apache.org/job/PreCommit-HADOOP-Build/6511/artifact/patchprocess/diffcheckstylehadoop-common.txt | | hadoop-common test log | https://builds.apache.org/job/PreCommit-HADOOP-Build/6511/artifact/patchprocess/testrun_hadoop-common.txt | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/6511/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/6511/console | This message was automatically generated. Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Attachments: HADOOP-11920-HDFS-7285-02.patch, HADOOP-11920-HDFS-7285-v4.patch, HADOOP-11920-HDFS-7285-v5.patch, HADOOP-11920-v1.patch, HADOOP-11920-v2.patch, HADOOP-11920-v3.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11920) Refactor some codes for erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14528408#comment-14528408 ] Kai Zheng commented on HADOOP-11920: HADOOP-11921 was opened to enhance the tests separately. Refactor some codes for erasure coders -- Key: HADOOP-11920 URL: https://issues.apache.org/jira/browse/HADOOP-11920 Project: Hadoop Common Issue Type: Sub-task Components: io Reporter: Kai Zheng Assignee: Kai Zheng Attachments: HADOOP-11920-v1.patch, HADOOP-11920-v2.patch While working on native erasure coders and also HADOOP-11847, it was found in some chances better to refine a little bit of codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)