[
https://issues.apache.org/jira/browse/HDFS-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17928777#comment-17928777
]
ASF GitHub Bot commented on HDFS-17732:
---------------------------------------
LiuGuH opened a new pull request, #7414:
URL: https://github.com/apache/hadoop/pull/7414
…Storages in PendingReconstructionBlocks
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines:
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g.,
'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
Now when a block is commit , it should add expectedStorages into
PendingReconstructionBlocks if the ibr is not sent from expected locations.
And for ec groups , only blkStriped.getRealTotalBlockNum() ==
expectedStorages.length will be added into PendingReconstructionBlocks.
But the following situations need to be considered：
(1)A EC file with XOR-2-1-1024k，when we write lower than 1024k into the
file，close it.
(2)Now The file has only one ec block group, indics=(0,1,2) and suppose only
0 replica ibr is received. Then the block group commit and will be completed.
(3) When block group commit , blkStriped.getRealTotalBlockNum()=1
expectedStorages.length=3 , so it can not add EC block expectedStorages in
PendingReconstructionBlocks
(4) This will result in the ec group reconstruction
(5) The 1 and 2 replicas ibr is recived.
(6) This maybe lead two same block group indics on same datanode if
reconstruction choose the same datanode
> addExpectedReplicasToPending should add EC block expectedStorages in
> PendingReconstructionBlocks
> --------------------------------------------------------------------------------------------------
>
> Key: HDFS-17732
> URL: https://issues.apache.org/jira/browse/HDFS-17732
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: liuguanghua
> Priority: Major
>
> Now when a block is commit , it should add expectedStorages into
> PendingReconstructionBlocks if the ibr is not sent from expected locations.
> And for ec groups , only blkStriped.getRealTotalBlockNum() ==
> expectedStorages.length will be added into PendingReconstructionBlocks.
>
> But the following situations need to be considered：
> (1)A EC file with XOR-2-1-1024k，when we write lower than 1024k into the
> file，close it.
> (2)Now The file has only one ec block group, indics=(0,1,2) and suppose only
> 0 replica ibr is received. Then the block group commit and will be
> completed.
> (3) When block group commit , blkStriped.getRealTotalBlockNum()=1
> expectedStorages.length=3 , so it can not add EC block expectedStorages in
> PendingReconstructionBlocks
> (4) This will result in the ec group reconstruction
> (5) The 1 and 2 replicas ibr is recived.
> (6) This maybe lead two same block group indics on same datanode if
> reconstruction choose the same datanode
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
