[
https://issues.apache.org/jira/browse/HDFS-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366760#comment-14366760
]
Jing Zhao commented on HDFS-7369:
---------------------------------
Some questions and comments:
# A question on BlockECRecoveryInfo and BlockECRecoveryCommand: for a (m,k) EC
schema, I guess a DN needs to know these information for recovery: 1) block ID
and block pool ID, 2) new generation stamp, 3) DNs with healthy blocks and
their corresponding index, 4) target DNs and storages, and 5) EC schema
information. Looks like we currently do not have 2) and 5).
# In BlockECRecoveryCommand, can we use DatanodeStorageInfo directly instead of
separating it into DatanodeInfo, StorageType, and storage ID? Also, these
blocks for recovery may not belong to the same block pool. Thus we may need to
use ExtendedBlock here.
# Minor: for BlockECRecoveryCommand, instead of using two dimension arrays, it
may be more clear to use a structure like {{BlockRecoveryCommand}}, i.e., to
wrap information for one striped block into a single class.
# Minor: need to fix the javadoc of BlockRecoveryWork (srcNodes and targets
cannot be resolved). Also need to fix javadoc for the BlockECRecoveryCommand
class.
# Minor: {{BlockManager#chooseSourceDatanodes}}'s javadoc needs reformat. Also
{{missingBlockIndices}}'s javadoc is missing.
# Nit-pick: the change of imports in DatanodeDescriptor is unnecessary
> Erasure coding: distribute recovery work for striped blocks to DataNode
> -----------------------------------------------------------------------
>
> Key: HDFS-7369
> URL: https://issues.apache.org/jira/browse/HDFS-7369
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Zhe Zhang
> Assignee: Zhe Zhang
> Attachments: HDFS-7369-000-part1.patch, HDFS-7369-000-part2.patch,
> HDFS-7369-001.patch, HDFS-7369-002.patch, HDFS-7369-003.patch
>
>
> This JIRA updates NameNode to handle background / offline recovery of erasure
> coded blocks. It includes 2 parts:
> # Extend {{UnderReplicatedBlocks}} to recognize EC blocks and insert them to
> appropriate priority levels.
> # Update {{ReplicationMonitor}} to distinguish block codec tasks and send a
> new DataNode command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
