[
https://issues.apache.org/jira/browse/HDFS-7715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14486240#comment-14486240
]
Kai Zheng commented on HDFS-7715:
---------------------------------
[~jack_liuquan],
A quick look at the test codes:
1. The major work is well done. Great!
2. Please have a new class like {{TestHitchhikerRawCoderBase}}, and move the
basic utility codes there; and for the 3 modes, have 3 unit test classes
respectively. For now, we just need something like
{{TestHitchhikerXORRawCoder}}. Please remove test codes for other modes, since
for the first patch, we only need to implement the first mode.
3. Why you prefer to use {{byte[]}} and come up your own utility functions for
that? Please note we need to support the 3 three versions: {{byte[]}},
{{ByteBuffer}} and {{ECChunk}}. Erasure coder caller will use the 3rd version
in most cases. The class will allow us to play some tricks overriding something
when necessary, and also, for performance gain it may wrap a ByteBuffer from
off-heap.
4. As discussed offline with you, it was said HH would be nothing too different
from caller's point of view. So please keep consistent contracts and test
methods with other coders. In some time we may have some comparing between HH
and RS in a simple loop.
> Implement the Hitchhiker erasure coding algorithm
> -------------------------------------------------
>
> Key: HDFS-7715
> URL: https://issues.apache.org/jira/browse/HDFS-7715
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Zhe Zhang
> Assignee: jack liuquan
> Attachments: 7715-hitchhikerXOR-v2-testcode.patch,
> 7715-hitchhikerXOR-v2.patch, HDFS-7715-hhxor-decoder.patch,
> HDFS-7715-hhxor-encoder.patch
>
>
> [Hitchhiker |
> http://www.eecs.berkeley.edu/~nihar/publications/Hitchhiker_SIGCOMM14.pdf] is
> a new erasure coding algorithm developed as a research project at UC
> Berkeley. It has been shown to reduce network traffic and disk I/O by 25%-45%
> during data reconstruction. This JIRA aims to introduce Hitchhiker to the
> HDFS-EC framework, as one of the pluggable codec algorithms.
> The existing implementation is based on HDFS-RAID.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
