[
https://issues.apache.org/jira/browse/HDFS-9833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307142#comment-15307142
]
Kai Zheng commented on HDFS-9833:
---------------------------------
Thanks [~rakeshr] for the update. It looks much close now. I have two more
comments:
1. Would it be good to use {{StripedReconstructionInfo}} in the
{{StripedReconstructor}} family? I mean, we can convert
{{BlockECReconstructionInfo}} to StripedReconstructionInfo to pass down, and we
can use it in the base constructor and so on.
{code}
+ public StripedBlockChecksumReconstructor(ErasureCodingWorker worker,
+ ExtendedBlock blockGroup, DatanodeInfo[] srcNodes, byte[] liveIndices,
+ int[] targetIndices, ErasureCodingPolicy ecPolicy,
+ DataOutputBuffer checksumWriter) throws IOException
{code}
2. How about {{ECBlockInfo}} => {{LiveBlockInfo}}, and
{{setChecksumProperties}} => {{setOrCheckChecksumProperties}}?
+1 once addressed.
> Erasure coding: recomputing block checksum on the fly by reconstructing the
> missed/corrupt block data
> -----------------------------------------------------------------------------------------------------
>
> Key: HDFS-9833
> URL: https://issues.apache.org/jira/browse/HDFS-9833
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Kai Zheng
> Assignee: Rakesh R
> Labels: hdfs-ec-3.0-must-do
> Attachments: HDFS-9833-00-draft.patch, HDFS-9833-01.patch,
> HDFS-9833-02.patch, HDFS-9833-03.patch, HDFS-9833-04.patch,
> HDFS-9833-05.patch, HDFS-9833-06.patch, HDFS-9833-07.patch
>
>
> As discussed in HDFS-8430 and HDFS-9694, to compute striped file checksum
> even some of striped blocks are missed, we need to consider recomputing block
> checksum on the fly for the missed/corrupt blocks. To recompute the block
> checksum, the block data needs to be reconstructed by erasure decoding, and
> the main needed codes for the block reconstruction could be borrowed from
> HDFS-9719, the refactoring of the existing {{ErasureCodingWorker}}. In EC
> worker, reconstructed blocks need to be written out to target datanodes, but
> here in this case, the remote writing isn't necessary, as the reconstructed
> block data is only used to recompute the checksum.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
