[
https://issues.apache.org/jira/browse/HDFS-7652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310154#comment-14310154
]
Jing Zhao commented on HDFS-7652:
---------------------------------
Thanks for working on this, Zhe! Basically to process a report for a stripped
block we always need these two pieces of information: 1) the reported block
that contains the block id seen by DN, and 2) the corresponding
BlockInfo(Stripped) object stored in the blocksMap. The former is necessary to
identify the internal index of the reported block inside of the block group.
This part of logic has been included in the current patch in HDFS-7716. For
this jira, I think {{BlockManager#getStoredBlock}} should be the place to add
the logic identifying if the reported block points to a contiguous block or a
stripped block group (based on the id and also map lookup result).
> Process block reports for erasure coded blocks
> ----------------------------------------------
>
> Key: HDFS-7652
> URL: https://issues.apache.org/jira/browse/HDFS-7652
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Zhe Zhang
> Assignee: Zhe Zhang
> Attachments: HDFS-7652.001.patch
>
>
> HDFS-7339 adds support in NameNode for persisting block groups. For memory
> efficiency, erasure coded blocks under the striping layout are not stored in
> {{BlockManager#blocksMap}}. Instead, entire block groups are stored in
> {{BlockGroupManager#blockGroups}}. When a block report arrives from the
> DataNode, it should be processed under the block group that it belongs to.
> The following naming protocol is used to calculate the group of a given block:
> {code}
> * HDFS-EC introduces a hierarchical protocol to name blocks and groups:
> * Contiguous: {reserved block IDs | flag | block ID}
> * Striped: {reserved block IDs | flag | block group ID | index in group}
> *
> * Following n bits of reserved block IDs, The (n+1)th bit in an ID
> * distinguishes contiguous (0) and striped (1) blocks. For a striped block,
> * bits (n+2) to (64-m) represent the ID of its block group, while the last m
> * bits represent its index of the group. The value m is determined by the
> * maximum number of blocks in a group (MAX_BLOCKS_IN_GROUP).
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
