Kai Zheng created HDFS-7345:
-------------------------------
Summary: Local Reconstruction Codes (LRC)
Key: HDFS-7345
URL: https://issues.apache.org/jira/browse/HDFS-7345
Project: Hadoop HDFS
Issue Type: Sub-task
Affects Versions: HDFS-EC
Reporter: Kai Zheng
Assignee: Kai Zheng
HDFS-7285 proposes to support Erasure Coding inside HDFS, supports multiple
Erasure Coding codecs via pluggable framework and implements Reed Solomon code
by default. This is to support a more advanced coding mechanism, Local
Reconstruction Codes (LRC). As discussed in the paper
(https://www.usenix.org/system/files/conference/atc12/atc12-final181_0.pdf),
LRC reduces the number of erasure coding fragments that need to be read when
reconstructing data fragments that are offline, while still keeping the storage
overhead low. The important benefits of LRC are that it reduces the bandwidth
and I/Os required for repair reads over prior codes, while still allowing a
significant reduction in storage overhead. Intel ISA library also supports LRC
in its update and can also be leveraged. The implementation would also consider
how to distribute the calculating of local and global parity blocks to other
relevant DataNodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
