[jira] [Commented] (HDFS-8223) Should calculate checksum for parity blocks in DFSStripedOutputStream
[
https://issues.apache.org/jira/browse/HDFS-8223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938488#comment-14938488
]
Hudson commented on HDFS-8223:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2408 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2408/])
HDFS-8223. Should calculate checksum for parity blocks in (zhezhang: rev
3f2c6938f1d157ff587826bd91b09906e60b53f4)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSOutputSummer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java
> Should calculate checksum for parity blocks in DFSStripedOutputStream
> -
>
> Key: HDFS-8223
> URL: https://issues.apache.org/jira/browse/HDFS-8223
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Yi Liu
>Assignee: Yi Liu
> Fix For: HDFS-7285
>
> Attachments: HDFS-8223.001.patch, HDFS-8223.rebase.patch
>
>
> {quote}
> DFSPacket p = createPacket(packetSize, chunksPerPacket,
> streamer.getBytesCurBlock(),
> streamer.getAndIncCurrentSeqno(), false);
> int maxBytesToPacket = p.getMaxChunks() * bytesPerChecksum;
> int toWrite = byteBuffer.remaining() > maxBytesToPacket ?
> maxBytesToPacket: byteBuffer.remaining();
> p.writeData(byteBuffer, toWrite);
> streamer.incBytesCurBlock(toWrite);
> packets.add(p);
> {quote}
> In {{DFSStripedOutputStream}}, have not calculated checksum for parity blocks
> in {{DFSStripedOutputStream}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8223) Should calculate checksum for parity blocks in DFSStripedOutputStream
[
https://issues.apache.org/jira/browse/HDFS-8223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938680#comment-14938680
]
Hudson commented on HDFS-8223:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #465 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/465/])
HDFS-8223. Should calculate checksum for parity blocks in (zhezhang: rev
3f2c6938f1d157ff587826bd91b09906e60b53f4)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSOutputSummer.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
> Should calculate checksum for parity blocks in DFSStripedOutputStream
> -
>
> Key: HDFS-8223
> URL: https://issues.apache.org/jira/browse/HDFS-8223
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Yi Liu
>Assignee: Yi Liu
> Fix For: HDFS-7285
>
> Attachments: HDFS-8223.001.patch, HDFS-8223.rebase.patch
>
>
> {quote}
> DFSPacket p = createPacket(packetSize, chunksPerPacket,
> streamer.getBytesCurBlock(),
> streamer.getAndIncCurrentSeqno(), false);
> int maxBytesToPacket = p.getMaxChunks() * bytesPerChecksum;
> int toWrite = byteBuffer.remaining() > maxBytesToPacket ?
> maxBytesToPacket: byteBuffer.remaining();
> p.writeData(byteBuffer, toWrite);
> streamer.incBytesCurBlock(toWrite);
> packets.add(p);
> {quote}
> In {{DFSStripedOutputStream}}, have not calculated checksum for parity blocks
> in {{DFSStripedOutputStream}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8223) Should calculate checksum for parity blocks in DFSStripedOutputStream
[
https://issues.apache.org/jira/browse/HDFS-8223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938111#comment-14938111
]
Hudson commented on HDFS-8223:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #439 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/439/])
HDFS-8223. Should calculate checksum for parity blocks in (zhezhang: rev
3f2c6938f1d157ff587826bd91b09906e60b53f4)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSOutputSummer.java
> Should calculate checksum for parity blocks in DFSStripedOutputStream
> -
>
> Key: HDFS-8223
> URL: https://issues.apache.org/jira/browse/HDFS-8223
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Yi Liu
>Assignee: Yi Liu
> Fix For: HDFS-7285
>
> Attachments: HDFS-8223.001.patch, HDFS-8223.rebase.patch
>
>
> {quote}
> DFSPacket p = createPacket(packetSize, chunksPerPacket,
> streamer.getBytesCurBlock(),
> streamer.getAndIncCurrentSeqno(), false);
> int maxBytesToPacket = p.getMaxChunks() * bytesPerChecksum;
> int toWrite = byteBuffer.remaining() > maxBytesToPacket ?
> maxBytesToPacket: byteBuffer.remaining();
> p.writeData(byteBuffer, toWrite);
> streamer.incBytesCurBlock(toWrite);
> packets.add(p);
> {quote}
> In {{DFSStripedOutputStream}}, have not calculated checksum for parity blocks
> in {{DFSStripedOutputStream}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8223) Should calculate checksum for parity blocks in DFSStripedOutputStream
[
https://issues.apache.org/jira/browse/HDFS-8223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937891#comment-14937891
]
Hudson commented on HDFS-8223:
--
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #473 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/473/])
HDFS-8223. Should calculate checksum for parity blocks in (zhezhang: rev
3f2c6938f1d157ff587826bd91b09906e60b53f4)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSOutputSummer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java
> Should calculate checksum for parity blocks in DFSStripedOutputStream
> -
>
> Key: HDFS-8223
> URL: https://issues.apache.org/jira/browse/HDFS-8223
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Yi Liu
>Assignee: Yi Liu
> Fix For: HDFS-7285
>
> Attachments: HDFS-8223.001.patch, HDFS-8223.rebase.patch
>
>
> {quote}
> DFSPacket p = createPacket(packetSize, chunksPerPacket,
> streamer.getBytesCurBlock(),
> streamer.getAndIncCurrentSeqno(), false);
> int maxBytesToPacket = p.getMaxChunks() * bytesPerChecksum;
> int toWrite = byteBuffer.remaining() > maxBytesToPacket ?
> maxBytesToPacket: byteBuffer.remaining();
> p.writeData(byteBuffer, toWrite);
> streamer.incBytesCurBlock(toWrite);
> packets.add(p);
> {quote}
> In {{DFSStripedOutputStream}}, have not calculated checksum for parity blocks
> in {{DFSStripedOutputStream}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8223) Should calculate checksum for parity blocks in DFSStripedOutputStream
[
https://issues.apache.org/jira/browse/HDFS-8223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937637#comment-14937637
]
Hudson commented on HDFS-8223:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #1203 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/1203/])
HDFS-8223. Should calculate checksum for parity blocks in (zhezhang: rev
3f2c6938f1d157ff587826bd91b09906e60b53f4)
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSOutputSummer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
> Should calculate checksum for parity blocks in DFSStripedOutputStream
> -
>
> Key: HDFS-8223
> URL: https://issues.apache.org/jira/browse/HDFS-8223
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Yi Liu
>Assignee: Yi Liu
> Fix For: HDFS-7285
>
> Attachments: HDFS-8223.001.patch, HDFS-8223.rebase.patch
>
>
> {quote}
> DFSPacket p = createPacket(packetSize, chunksPerPacket,
> streamer.getBytesCurBlock(),
> streamer.getAndIncCurrentSeqno(), false);
> int maxBytesToPacket = p.getMaxChunks() * bytesPerChecksum;
> int toWrite = byteBuffer.remaining() > maxBytesToPacket ?
> maxBytesToPacket: byteBuffer.remaining();
> p.writeData(byteBuffer, toWrite);
> streamer.incBytesCurBlock(toWrite);
> packets.add(p);
> {quote}
> In {{DFSStripedOutputStream}}, have not calculated checksum for parity blocks
> in {{DFSStripedOutputStream}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8223) Should calculate checksum for parity blocks in DFSStripedOutputStream
[
https://issues.apache.org/jira/browse/HDFS-8223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937388#comment-14937388
]
Hudson commented on HDFS-8223:
--
FAILURE: Integrated in Hadoop-trunk-Commit #8548 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/8548/])
HDFS-8223. Should calculate checksum for parity blocks in (zhezhang: rev
3f2c6938f1d157ff587826bd91b09906e60b53f4)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSOutputSummer.java
> Should calculate checksum for parity blocks in DFSStripedOutputStream
> -
>
> Key: HDFS-8223
> URL: https://issues.apache.org/jira/browse/HDFS-8223
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Yi Liu
>Assignee: Yi Liu
> Fix For: HDFS-7285
>
> Attachments: HDFS-8223.001.patch, HDFS-8223.rebase.patch
>
>
> {quote}
> DFSPacket p = createPacket(packetSize, chunksPerPacket,
> streamer.getBytesCurBlock(),
> streamer.getAndIncCurrentSeqno(), false);
> int maxBytesToPacket = p.getMaxChunks() * bytesPerChecksum;
> int toWrite = byteBuffer.remaining() > maxBytesToPacket ?
> maxBytesToPacket: byteBuffer.remaining();
> p.writeData(byteBuffer, toWrite);
> streamer.incBytesCurBlock(toWrite);
> packets.add(p);
> {quote}
> In {{DFSStripedOutputStream}}, have not calculated checksum for parity blocks
> in {{DFSStripedOutputStream}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8223) Should calculate checksum for parity blocks in DFSStripedOutputStream
[
https://issues.apache.org/jira/browse/HDFS-8223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937191#comment-14937191
]
Hudson commented on HDFS-8223:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #2379 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2379/])
HDFS-8223. Should calculate checksum for parity blocks in (zhezhang: rev
3f2c6938f1d157ff587826bd91b09906e60b53f4)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSOutputSummer.java
> Should calculate checksum for parity blocks in DFSStripedOutputStream
> -
>
> Key: HDFS-8223
> URL: https://issues.apache.org/jira/browse/HDFS-8223
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Yi Liu
>Assignee: Yi Liu
> Fix For: HDFS-7285
>
> Attachments: HDFS-8223.001.patch, HDFS-8223.rebase.patch
>
>
> {quote}
> DFSPacket p = createPacket(packetSize, chunksPerPacket,
> streamer.getBytesCurBlock(),
> streamer.getAndIncCurrentSeqno(), false);
> int maxBytesToPacket = p.getMaxChunks() * bytesPerChecksum;
> int toWrite = byteBuffer.remaining() > maxBytesToPacket ?
> maxBytesToPacket: byteBuffer.remaining();
> p.writeData(byteBuffer, toWrite);
> streamer.incBytesCurBlock(toWrite);
> packets.add(p);
> {quote}
> In {{DFSStripedOutputStream}}, have not calculated checksum for parity blocks
> in {{DFSStripedOutputStream}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8223) Should calculate checksum for parity blocks in DFSStripedOutputStream
[
https://issues.apache.org/jira/browse/HDFS-8223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14510565#comment-14510565
]
Yi Liu commented on HDFS-8223:
--
Thanks Jing for review/rebase/commit !
> Should calculate checksum for parity blocks in DFSStripedOutputStream
> -
>
> Key: HDFS-8223
> URL: https://issues.apache.org/jira/browse/HDFS-8223
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Yi Liu
>Assignee: Yi Liu
> Fix For: HDFS-7285
>
> Attachments: HDFS-8223.001.patch, HDFS-8223.rebase.patch
>
>
> {quote}
> DFSPacket p = createPacket(packetSize, chunksPerPacket,
> streamer.getBytesCurBlock(),
> streamer.getAndIncCurrentSeqno(), false);
> int maxBytesToPacket = p.getMaxChunks() * bytesPerChecksum;
> int toWrite = byteBuffer.remaining() > maxBytesToPacket ?
> maxBytesToPacket: byteBuffer.remaining();
> p.writeData(byteBuffer, toWrite);
> streamer.incBytesCurBlock(toWrite);
> packets.add(p);
> {quote}
> In {{DFSStripedOutputStream}}, have not calculated checksum for parity blocks
> in {{DFSStripedOutputStream}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8223) Should calculate checksum for parity blocks in DFSStripedOutputStream
[
https://issues.apache.org/jira/browse/HDFS-8223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14509985#comment-14509985
]
Jing Zhao commented on HDFS-8223:
-
Thanks for fixing this, Yi! The patch looks good to me. +1
Besides, the patch needs rebase. You can do it while committing.
> Should calculate checksum for parity blocks in DFSStripedOutputStream
> -
>
> Key: HDFS-8223
> URL: https://issues.apache.org/jira/browse/HDFS-8223
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Yi Liu
>Assignee: Yi Liu
> Attachments: HDFS-8223.001.patch
>
>
> {quote}
> DFSPacket p = createPacket(packetSize, chunksPerPacket,
> streamer.getBytesCurBlock(),
> streamer.getAndIncCurrentSeqno(), false);
> int maxBytesToPacket = p.getMaxChunks() * bytesPerChecksum;
> int toWrite = byteBuffer.remaining() > maxBytesToPacket ?
> maxBytesToPacket: byteBuffer.remaining();
> p.writeData(byteBuffer, toWrite);
> streamer.incBytesCurBlock(toWrite);
> packets.add(p);
> {quote}
> In {{DFSStripedOutputStream}}, have not calculated checksum for parity blocks
> in {{DFSStripedOutputStream}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8223) Should calculate checksum for parity blocks in DFSStripedOutputStream
[
https://issues.apache.org/jira/browse/HDFS-8223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14507153#comment-14507153
]
Yi Liu commented on HDFS-8223:
--
Will update the patch later.
> Should calculate checksum for parity blocks in DFSStripedOutputStream
> -
>
> Key: HDFS-8223
> URL: https://issues.apache.org/jira/browse/HDFS-8223
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Yi Liu
>Assignee: Yi Liu
>
> {quote}
> DFSPacket p = createPacket(packetSize, chunksPerPacket,
> streamer.getBytesCurBlock(),
> streamer.getAndIncCurrentSeqno(), false);
> int maxBytesToPacket = p.getMaxChunks() * bytesPerChecksum;
> int toWrite = byteBuffer.remaining() > maxBytesToPacket ?
> maxBytesToPacket: byteBuffer.remaining();
> p.writeData(byteBuffer, toWrite);
> streamer.incBytesCurBlock(toWrite);
> packets.add(p);
> {quote}
> In {{DFSStripedOutputStream}}, have not calculated checksum for parity blocks
> in {{DFSStripedOutputStream}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
