[
https://issues.apache.org/jira/browse/HDFS-10460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15341405#comment-15341405
]
Kai Zheng commented on HDFS-10460:
----------------------------------
Thanks Rakesh for the explanation. I understood the situation and we need an
extra parameter, what I suggested was like changing as follows:
1) Not doing this: {{block.setNumBytes(getRemaining())}}, or leave the numBytes
of block unchanged as it is, because for block group, we need to know the
original numBytes or actualNumBytes of the block group in datanode side for
reconstruction.
2) Add requestedLength parameter instead, passing it to datanode side to do the
checksum computing.
3) So the change of the protocol message would be like:
{code}
message OpBlockGroupChecksumProto {
repeated hadoop.common.TokenProto blockTokens = 3;
required ErasureCodingPolicyProto ecPolicy = 4;
repeated uint32 blockIndices = 5;
+ required uint64 requestedNumBytes = 6;
}
{code}
> Erasure Coding: Recompute block checksum for a particular range less than
> file size on the fly by reconstructing missed block
> -----------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-10460
> URL: https://issues.apache.org/jira/browse/HDFS-10460
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Reporter: Rakesh R
> Assignee: Rakesh R
> Attachments: HDFS-10460-00.patch, HDFS-10460-01.patch
>
>
> This jira is HDFS-9833 follow-on task to address reconstructing block and
> then recalculating block checksum for a particular range query.
> For example,
> {code}
> // create a file 'stripedFile1' with fileSize = cellSize * numDataBlocks =
> 65536 * 6 = 393216
> FileChecksum stripedFileChecksum = getFileChecksum(stripedFile1, 10, true);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
