[
https://issues.apache.org/jira/browse/HDFS-7345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14207583#comment-14207583
]
Kai Zheng commented on HDFS-7345:
---------------------------------
Anyway how it goes for the IP issue, I would continue with the relevant
prototype keeping the code in mind, as LRC is a typical case and we should make
sure the pluggable codec framework also works for that, even though the
resultant LRC codec plugin maybe eventually can't be contributed to the
community. In that case, customers can choose to resolve the IP issue with MS
and download/deploy the plugin for their own needs.
> Local Reconstruction Codes (LRC)
> --------------------------------
>
> Key: HDFS-7345
> URL: https://issues.apache.org/jira/browse/HDFS-7345
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Kai Zheng
> Assignee: Kai Zheng
>
> HDFS-7285 proposes to support Erasure Coding inside HDFS, supports multiple
> Erasure Coding codecs via pluggable framework and implements Reed Solomon
> code by default. This is to support a more advanced coding mechanism, Local
> Reconstruction Codes (LRC). As discussed in the paper
> (https://www.usenix.org/system/files/conference/atc12/atc12-final181_0.pdf),
> LRC reduces the number of erasure coding fragments that need to be read when
> reconstructing data fragments that are offline, while still keeping the
> storage overhead low. The important benefits of LRC are that it reduces the
> bandwidth and I/Os required for repair reads over prior codes, while still
> allowing a significant reduction in storage overhead. Intel ISA library also
> supports LRC in its update and can also be leveraged. The implementation
> would also consider how to distribute the calculating of local and global
> parity blocks to other relevant DataNodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
