[
https://issues.apache.org/jira/browse/HADOOP-11828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488623#comment-15488623
]
Kai Zheng commented on HADOOP-11828:
------------------------------------
Hi [~rashmikv],
Sorry for not seeing your question and the late reply. By compatible or
inter-operable, I mean the encoded data by a coder can be decoded by another
coder. The new Java coder is compatible with the native ISA-L coder in this
sense. In some environment in your clients you may not have Hadoop native setup
so only pure Java solution will do the work, but your other clients and the
cluster use Hadoop native so the native ISA-L based coder can work for better
performance. In other words, it's possible to use more than one coder
implementations in a cluster and its clients, so the requirement for such
implementation should be compatible with others, otherwise the data will be
messy.
> Implement the Hitchhiker erasure coding algorithm
> -------------------------------------------------
>
> Key: HADOOP-11828
> URL: https://issues.apache.org/jira/browse/HADOOP-11828
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 3.0.0-alpha1
> Reporter: Zhe Zhang
> Assignee: jack liuquan
> Fix For: 3.0.0-alpha1
>
> Attachments: 7715-hitchhikerXOR-v2-testcode.patch,
> 7715-hitchhikerXOR-v2.patch, HADOOP-11828-hitchhikerXOR-V3.patch,
> HADOOP-11828-hitchhikerXOR-V4.patch, HADOOP-11828-hitchhikerXOR-V5.patch,
> HADOOP-11828-hitchhikerXOR-V6.patch, HADOOP-11828-hitchhikerXOR-V7.patch,
> HADOOP-11828-v8.patch, HDFS-7715-hhxor-decoder.patch,
> HDFS-7715-hhxor-encoder.patch
>
>
> [Hitchhiker |
> http://www.eecs.berkeley.edu/~nihar/publications/Hitchhiker_SIGCOMM14.pdf] is
> a new erasure coding algorithm developed as a research project at UC
> Berkeley. It has been shown to reduce network traffic and disk I/O by 25%-45%
> during data reconstruction while retaining the same storage capacity and
> failure tolerance capability as RS codes. This JIRA aims to introduce
> Hitchhiker to the HDFS-EC framework, as one of the pluggable codec algorithms.
> The existing implementation is based on HDFS-RAID.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
