[
https://issues.apache.org/jira/browse/HDFS-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932176#comment-17932176
]
ASF GitHub Bot commented on HDFS-17732:
---------------------------------------
LiuGuH commented on PR #7414:
URL: https://github.com/apache/hadoop/pull/7414#issuecomment-2696498082
@Hexiaoqiao @ZanderXu @tomscut , Hello Sir , please reivew this PR if you
have time . Thanks very much
> addExpectedReplicasToPending should add EC block expectedStorages in
> PendingReconstructionBlocks
> ------------------------------------------------------------------------------------------------
>
> Key: HDFS-17732
> URL: https://issues.apache.org/jira/browse/HDFS-17732
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: liuguanghua
> Assignee: liuguanghua
> Priority: Major
> Labels: pull-request-available
>
> Now when a block commit , it should add expectedStorages into
> PendingReconstructionBlocks if the ibr is not sent from expected locations.
> And for a ec block group , only blkStriped.getRealTotalBlockNum() ==
> expectedStorages.length will be added into PendingReconstructionBlocks.
> The problem is that a samll file with ec, it may always
> blkStriped.getRealTotalBlockNum() < expectedStorages.length if the file size
> < ecPolicy.getNumDataUnits()*ecPolicy.getCellSize().
> The following situations need to be considered：
> (1) A EC file with XOR-2-1-1024k，when we write lower than 1024k into the
> file，close it.
> (2) Now The file has only one ec block group, indics=(0,1,2) and suppose only
> 0 replica ibr is received and 2 replica ibr is delayed. The ec block group
> reaches minstorage and commit .
> (3) When block group commit , blkStriped.getRealTotalBlockNum()=1
> expectedStorages.length=3 , so it can not add this ec block group
> expectedStorages into PendingReconstructionBlocks.
> (4) The ec block group complete and only has minstorage and the the namenode
> generates ec reconstruncation task to datanode.
> (5) The 2 replicas ibr is recived.
> (6) Datanode reconstruction task finished and ibr sends.
> (7) (5) and (6) may in a same datanode
> For other ec policy, in (7) , on a same datanode will has two different ec
> block indic. For ec this is not expected and if it happens , in namenode will
> be AddBlockResult.REPLACED replaced the former ibr. And it maybe increases
> the ec reconstruncation task again.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
