[
https://issues.apache.org/jira/browse/HDFS-7912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355294#comment-14355294
]
Zhe Zhang commented on HDFS-7912:
---------------------------------
Thanks for the work Jing! I'm working on HDFS-7369 which is related to this
thought.
I haven't looked {{PendingReplicationBlocks}} in detail. For
{{UnderReplicatedBlocks}} I guess we might be able to use the original
interface? After all its main job is to quantify the risk level of all blocks.
Maybe we can use the existing {{add(Block block, int curReplicas, int
decomissionedReplicas, int expectedReplicas)}} method to express the risk level
of a striped block group? The current HDFS-7369 patch uses the following
calculation based on this
[reasoning|https://issues.apache.org/jira/browse/HDFS-7369?focusedCommentId=14318849&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14318849]:
{code}
- short expectedReplication = bc.getBlockReplication();
+ short expectedReplication = bc.isStriped()?
+ HdfsConstants.NUM_PARITY_BLOCKS + 1 :
+ bc.getBlockReplication();
NumberReplicas num = countNodes(block);
int numCurrentReplica = num.liveReplicas();
+ if (bc.isStriped()) {
+ numCurrentReplica -= (HdfsConstants.NUM_DATA_BLOCKS - 1);
+ if (numCurrentReplica < 0) {
+ numCurrentReplica = 0;
+ }
+ }
{code}
Of course explicitly tracking {{BlockInfo}} gives even richer info to calculate
the risk level, and I don't see a downside except for the code change itself.
> Erasure Coding: track BlockInfo instead of Block in UnderReplicatedBlocks and
> PendingReplicationBlocks
> ------------------------------------------------------------------------------------------------------
>
> Key: HDFS-7912
> URL: https://issues.apache.org/jira/browse/HDFS-7912
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-7912.000.patch
>
>
> Now with striped blocks and the design that uses a single BlockInfoStriped
> object to track all the corresponding blocks, we need to clearly distinguish
> the type Block and BlockInfo in BlockManager. Specifically, data structures
> like {{UnderReplicatedBlocks}} and {{PendingReplicationBlocks}} should track
> BlockInfo instead of Block in order to support striped block recovery.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
