[
https://issues.apache.org/jira/browse/HDFS-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14343517#comment-14343517
]
Zhe Zhang commented on HDFS-7839:
---------------------------------
Thanks Kai! The updated description is more accurate.
bq. Is there a little bit conflict or overlap between this and HDFS-7349 ?
Good catch. I read Vinay's patch again and found it has PB implementation as
well. I actually planned to use this JIRA to support a simplest EC zone so we
can have an initial PoC prototype. HDFS-7349 investigates the most suitable set
of commands. So maybe we can keep the simple PB and NN RPC implementation under
this JIRA; HDFS-7349 can extend it later.
bq. So would you add to include some hard coded values like NumDataBlocks,
NumParityBlocks and etc
Where do you suggest we add those? In the XAttr?
bq. For the new operation to create an EC zone, we need two parameters,
selected EC schema (from predefined list) and the target folder.
I planned to leave the schema parameter to HDFS-7866 but we can certainly add a
place holder (a short type) here
bq. Better to move the logic of isStrippedFile out of the manager into
FSDirectory.
{{FSDirectory}} does have a {{isStrippedFile}} method. Per Andrew's suggestion,
I'm consolidating all actual XAttr operations in the manager class
bq. For consistency, ErasureCodingManager ==> ECZoneManager.
{{ErasureCodingManager}} was named following the convention of
{{EncryptionZoneManager}}.
Addressed both minor points in the updated patch (will upload after soliciting
more feedbacks).
I verified that nested encryption zones are not supported. We should just
disallow it at this stage as well. An encryption zone on root is allowed
though. Maybe we should allow it here as well?
> Erasure coding: implement facilities in NameNode to create and manage EC zones
> ------------------------------------------------------------------------------
>
> Key: HDFS-7839
> URL: https://issues.apache.org/jira/browse/HDFS-7839
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Zhe Zhang
> Assignee: Zhe Zhang
> Attachments: HDFS-7839-000.patch, HDFS-7839-001.patch
>
>
> As a quick first step to facilitate initial development and testing,
> HDFS-7347 added EC configuration in file header as one storage policy. We
> have discussed and [concluded
> |https://issues.apache.org/jira/browse/HDFS-7285?focusedCommentId=14296210&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14296210]
> that EC configurations should be part of XAttr. This JIRA aims to add the
> basic EC XAttr structure. HDFS-7337 will add configurable and pluggable
> schema info.
> To summarize, this will focus on providing relevant facilities in NameNode
> side to create and manage EC zones. EC zone will associate, reference and
> store the needed schema name as XAttr. EC schema information will be
> persisted in NameNode elsewhere centrally.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
