Hi, Thanks for the reply, I'm new to this, I understand from the attachment in HDFS-8030[1], in the description of this document[2], the encoding is done offline, the blocks initially still exist as copies and are converted to Erasure coding form. I know erasure coding is mainly suitable for cold data.
[1]https://issues.apache.org/jira/browse/HDFS-8030 [2]https://issues.apache.org/jira/secure/attachment/12775826/HDFSErasureCodingPhaseII-20151204.pdf 从 Windows 版邮件发送 发件人: Ayush Saxena 发送时间: 2024年4月22日 19:06 收件人: 1278789...@qq.com 抄送: user@hadoop.apache.org 主题: Re: How is HDFS Erasure Coding Phase II now？ Hi, > Or is it just not developed to this point? It isn't developed & I don't think there is any effort going on in that direction > I learned that continuous layout can ensure the locality of file blocks How? Erasure Coding will have BlockGroups not just one Block, whether you write in a striped manner or in a Contiguous manner, it will spread over equal number of Datanodes based on the BPP, I am not sure if anything changes with locality, just by the way how EC Blocks are written. > , I have large files and write them once and read them many times. Erasure Coding in general was developed for storing Archival data, so you need to figure out how "many" is ok. -Ayush On Mon, 22 Apr 2024 at 15:56, zhangzhengli <1278789...@qq.com.invalid> wrote: Hi all, Since HDFS-8030, hdfs ec continuous layout has not developed much. Are there any difficulties? Or is it just not developed to this point? I learned that continuous layout can ensure the locality of file blocks, and I want to use this feature in near-data scenarios. For example, I have large files and write them once and read them many times. Any suggestions are appreciated 从 Windows 版邮件发送
