[
https://issues.apache.org/jira/browse/HDFS-7715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377598#comment-14377598
]
Kai Zheng commented on HDFS-7715:
---------------------------------
Took a quick at the large patch, and my comments so far are:
1. Why we need to change hadoop-common/pom file ?
2. Please clean up and refine your codes considering: 1) regarding public or
protected variables and methods; 2) coding style; 3) comments.
3. Maybe we can have a utility class for piggyback stuff to simplify the HH
coders.
My major concern is that we're implementing the algorithm and the 3 modes from
bottom, which might be avoided since in the underlying, it uses XOR and
Reed-Solomon calculation, so existing XOR and RS raw coders may be possibly
used. In this way HH coders can be much simplified, more important, native XOR
and RS raw coders can be utilized to benefit from the performance improvement.
To make it much easier for review, would you:
1. Attach a patch with only HH basics plus the most simple mode coder, so that
it's minimized to ease the understanding.
2. Please don't use zip format, attach the patch directly.
> Implement the Hitchhiker erasure coding algorithm
> -------------------------------------------------
>
> Key: HDFS-7715
> URL: https://issues.apache.org/jira/browse/HDFS-7715
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Zhe Zhang
> Assignee: jack liuquan
> Attachments: HDFS-7715.zip
>
>
> [Hitchhiker |
> http://www.eecs.berkeley.edu/~nihar/publications/Hitchhiker_SIGCOMM14.pdf] is
> a new erasure coding algorithm developed as a research project at UC
> Berkeley. It has been shown to reduce network traffic and disk I/O by 25%-45%
> during data reconstruction. This JIRA aims to introduce Hitchhiker to the
> HDFS-EC framework, as one of the pluggable codec algorithms.
> The existing implementation is based on HDFS-RAID.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
