[
https://issues.apache.org/jira/browse/HDFS-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495954#comment-14495954
]
Vinayakumar B commented on HDFS-7337:
-------------------------------------
Thanks for the update [~drankye].
Updated section clearly mentioned about how EC schemas can be made configurable.
One improvement I am thinking of is,
Instead of maintaining an xml at serverside, which needs to be updated and then
issue command to load from it, lets make admin's life simpler.
* we can have explicit commands to add/delete schemas from the shell itself.
* In case of adding new schemas, (this will not be used for the deletion)
*# If the schema is simple with few parameters, then it can be given as
argument to command in predefined structure(propably with comma separated).
*# Otherwise command can load the schemas from admin-side *locally*
maintained xml and send them via RPC to NameNode. In this argument will be the
file name to be loaded. By this way admin can create xml and execute command
from any machine using admin authentication.
* For deletion of schemas, separate command via separate RPC, with just schema
name(s) as param, can do the job. Of-course it should be deleted only if not
used.
So this way, XML is used just for the friendly way of specifying schemas.
NameNode doesn't need to depend on this.
Any thoughts?
> Configurable and pluggable Erasure Codec and schema
> ---------------------------------------------------
>
> Key: HDFS-7337
> URL: https://issues.apache.org/jira/browse/HDFS-7337
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Zhe Zhang
> Assignee: Kai Zheng
> Attachments: HDFS-7337-prototype-v1.patch,
> HDFS-7337-prototype-v2.zip, HDFS-7337-prototype-v3.zip,
> PluggableErasureCodec-v2.pdf, PluggableErasureCodec-v3.pdf,
> PluggableErasureCodec.pdf
>
>
> According to HDFS-7285 and the design, this considers to support multiple
> Erasure Codecs via pluggable approach. It allows to define and configure
> multiple codec schemas with different coding algorithms and parameters. The
> resultant codec schemas can be utilized and specified via command tool for
> different file folders. While design and implement such pluggable framework,
> it’s also to implement a concrete codec by default (Reed Solomon) to prove
> the framework is useful and workable. Separate JIRA could be opened for the
> RS codec implementation.
> Note HDFS-7353 will focus on the very low level codec API and implementation
> to make concrete vendor libraries transparent to the upper layer. This JIRA
> focuses on high level stuffs that interact with configuration, schema and etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
