[
https://issues.apache.org/jira/browse/HDFS-9603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15148127#comment-15148127
]
Kai Zheng commented on HDFS-9603:
---------------------------------
Thanks for the discussion, [~zhz] and [~kaisasak]!
bq. Could you help clarify if Hitchhiker needs the ECBlockGroup? Or only LRC
potentially needs it?
Not sure if better to make the major change to re-implement HH in raw coder
level. I'll look at the algorithm and codes closely later, and check with Rui
how to make the small fix for the mentioned issue to use the HH coder with
ECBlockGroup. I thought we'll need the high level abstract ion eventually, not
just LRC, but also for other advanced algorithms, though RS is the right one we
need for the near future.
bq. With LRC looking unlikely to be implemented before the release, I think we
should focus on ...
Sounds good to me. The priority of this issue and the aspect should be low. I
guess it's not bad if someone would work on it, unless conflicted. Note
ErasureCoder can be used and co-exist meanwhile with raw coders.
One thing to add is, HDFS-8430 and the family, for striped file checksum.
bq. It is reasonable to keep using raw coders for now because hflush/hsync is
important to reliable operation to HDFS.
Is there anything overhead to use ErasureCoder for the support of hflush/hsync?
I thought they're not related. hflush/hsync support should not couple with any
specific codec/coder.
> Erasure Coding: Use ErasureCoder to encode/decode a block group
> ---------------------------------------------------------------
>
> Key: HDFS-9603
> URL: https://issues.apache.org/jira/browse/HDFS-9603
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Rui Li
> Assignee: Rui Li
> Attachments: HDFS-9603.1.patch, HDFS-9603.2.patch, HDFS-9603.3.patch
>
>
> According to design, {{ErasureCoder}} is responsible to encode/decode a block
> group. Currently however, we directly use {{RawErasureCoder}} to do the work,
> e.g. in {{DFSStripedOutputStream}}. This task attempts to encapsulate
> {{RawErasureCoder}} to comply with the design.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
