[
https://issues.apache.org/jira/browse/HDFS-8494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577586#comment-14577586
]
Jing Zhao commented on HDFS-8494:
---------------------------------
Thanks for working on this, [~kaisasak]!
In general I'm not very sure if we need to remove the hard-coded chunk size at
this stage. But if we want to do this, we should record this information in
BlockInfoStriped so that BlockInfoStriped is self explained. I think the basic
idea is that we'd better decouple the logic of the blocks, which belong to the
storage level, from the files. This means if one day we separate the block
management out of the current namenode, the block manager service can provide
enough information for a reader to read the striped block.
For memory footprint, we can easily do some follow-on optimization to save this
extra 4 bytes. E.g., we can store all the existing combinations of (ecschema +
chunk size) somewhere and only record an id in each striped block.
> Remove hard-coded chunk size in favor of ECZone
> -----------------------------------------------
>
> Key: HDFS-8494
> URL: https://issues.apache.org/jira/browse/HDFS-8494
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: HDFS-7285
> Reporter: Kai Sasaki
> Assignee: Kai Sasaki
> Fix For: HDFS-7285
>
> Attachments: HDFS-8494-HDFS-7285-01.patch,
> HDFS-8494-HDFS-7285-02.patch
>
>
> It is necessary to remove hard-coded values inside NameNode configured in
> {{HdfsConstants}}. In this JIRA, we can remove {{chunkSize}} gracefully in
> favor of HDFS-8375.
> Because {{cellSize}} is now originally stored only in {{ErasureCodingZone}},
> {{BlockInfoStriped}} can receive {{cellSize}} in addition to {{ECSchema}}
> when its initialization.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
