[
https://issues.apache.org/jira/browse/HDFS-8120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495769#comment-14495769
]
Jing Zhao commented on HDFS-8120:
---------------------------------
Thanks for the explanation, Bo.
bq. When DFSStripedOutputStream requests a block group from NN, it also doesn't
know how many blocks in this block group will be empty. The client may just
wirte 1 byte, which makes most of the blocks in this block group empty.
I think this can still be handled following similar logic of DFSOutputStream:
# A new striped block is allocated only when there is data written to it
# When the client tries to close the file, NameNode learns the last block's
length from the "block commit" request. Based on this length, the NameNode can
compute the expected number of internal blocks, and the {{completeFile}} call
succeeds only when NN receives enough number of "block received" reports from
DNs.
Anyway, I will try to fix this in HDFS-8145, and we can continue the discussion
there.
> Erasure coding: created util class to analyze striped block groups
> ------------------------------------------------------------------
>
> Key: HDFS-8120
> URL: https://issues.apache.org/jira/browse/HDFS-8120
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Zhe Zhang
> Assignee: Zhe Zhang
> Attachments: HDFS-8120.000.patch, HDFS-8120.001.patch,
> HDFS-8120.002.patch, HDFS-8120.003.patch, HDFS-8120.004.patch,
> HDFS-8120.005.patch, HDFS-8120.006.patch
>
>
> The patch adds logic of calculating size of individual blocks in a striped
> block group.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
