[
https://issues.apache.org/jira/browse/HDFS-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17927895#comment-17927895
]
liuguanghua commented on HDFS-15359:
------------------------------------
Hello, Sir. I have some questions about addForStriped condition. Now in case
of erasure coding blocks, adds only in case there isn't any missing node.
Function addExpectedReplicasToPending() will add block into
pendingReconstruction.
addForStriped = blkStriped.getRealTotalBlockNum() == expectedStorages.length
Why should we decide based on addForStriped？ Can we add all expectedStorages
into pendingReconstruction like replicated block does？
In addition, XOR-2-1-1024k, if we only write 1024K into a ec file, the close()
, blkStriped.getRealTotalBlockNum() = 1 and expectedStorages.length =3 ,this
will lead this ec group make ECReconstrunctin Task in namenode first. And then
the later ibr is coming.
[~ayushtkn] [~weichiu]
> EC: Allow closing a file with committed blocks
> ----------------------------------------------
>
> Key: HDFS-15359
> URL: https://issues.apache.org/jira/browse/HDFS-15359
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: erasure-coding
> Affects Versions: 3.4.0
> Reporter: Ayush Saxena
> Assignee: Ayush Saxena
> Priority: Major
> Fix For: 3.4.0
>
> Attachments: HDFS-15359-01.patch, HDFS-15359-02.patch,
> HDFS-15359-03.patch, HDFS-15359-04.patch, HDFS-15359-05.patch
>
>
> Presently, {{dfs.namenode.file.close.num-committed-allowed}} is ignored in
> case of EC blocks. But in case of heavy loads, IBR's from Datanode may get
> delayed and cause the file write to fail. So, can allow EC files to close
> with blocks in committed state as REP files
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
