[
https://issues.apache.org/jira/browse/HDFS-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15057515#comment-15057515
]
GAO Rui commented on HDFS-7344:
-------------------------------
[~zhz] Thank you very much for your information. I am trying to draft a design
and then implement {{Converter}} which is mentioned in HDFS-7717. I think maybe
we should design as {{Converter}} distribute conversation tasks to several
{{ErasureCodingWorker}}, then actually these {{ErasureCodingWorker}} implement
conversation tasks. Could you share your opinions? Currently, if put Erasure
Coding to product clusters, {{Converter}} should be among the most used
functions. Lots of replication files would be converted to EC files. Without
{{Converter}}, we can only use distcp. {{Converter}} should could be much more
efficient, right?
> [umbrella] Erasure Coding worker and support in DataNode
> --------------------------------------------------------
>
> Key: HDFS-7344
> URL: https://issues.apache.org/jira/browse/HDFS-7344
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Reporter: Kai Zheng
> Assignee: Li Bo
> Attachments: ECWorker-design-v2.pdf, HDFS ECWorker Design.pdf,
> hdfs-ec-datanode.0108.zip, hdfs-ec-datanode.0108.zip
>
>
> According to HDFS-7285 and the design, this handles DataNode side extension
> and related support for Erasure Coding. More specifically, it implements
> {{ECWorker}}, which reconstructs lost blocks (in striping layout).
> It generally needs to restore BlockGroup and schema information from coding
> commands from NameNode or other entities, and construct specific coding work
> to execute. The required block reader, writer, either local or remote,
> encoder and decoder, will be implemented separately as sub-tasks.
> This JIRA will track all the linked sub-tasks, and is responsible for general
> discussions and integration for ECWorker. It won't resolve until all the
> related tasks are done.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
