[
https://issues.apache.org/jira/browse/HDFS-11152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15829676#comment-15829676
]
SammiChen commented on HDFS-11152:
----------------------------------
Hi [~andrew.wang]] I did some investigation. It seems EC policy ID is not
widely used. For examples,
1. For directory, EC policy is saved as XAttr using policy name(policy name
doesn't include policy ID). When directory is saved to FsImage, all its XAttrs
are saved including EC policy name.
2. For file, EC policy ID is saved in head. When file is saved to FsImage, EC
policy ID is saved in replication field.
3. The {{setErasureCodingPolicy}} API, user will provide a
{{ErasureCodingPolicy}} parameter, currently only the schema and cell size is
used to check if it's a legitimate EC policy, policy ID is not involved in
parameter checking. Because currently {{setErasureCodingPolicy}} can only
applies to directory. And only policy name is saved on directory. I will
enforce the policy ID check in HDFS-10996 implementation which specify
per-file policy when create file.
4. File read/write. Double checked {{DFSStripedInputStream}} and
{{DFSStripedOutputStream}}, policy ID is not used at all. The schema and
cellsize are used.
So, later I plan to upload a new patch, adding Precondition checks on FsImage
file loading only. Because other spaces use policy name, schema and cell size.
> Start erasure coding policy ID number from 1 instead of 0 to void potential
> unexpected errors
> ---------------------------------------------------------------------------------------------
>
> Key: HDFS-11152
> URL: https://issues.apache.org/jira/browse/HDFS-11152
> Project: Hadoop HDFS
> Issue Type: Task
> Affects Versions: 3.0.0-alpha1
> Reporter: SammiChen
> Assignee: SammiChen
> Labels: hdfs-ec-3.0-must-do
> Attachments: HDFS-11152-v1.patch
>
>
> This task will change erasure coding policy ID number starting from 1 instead
> of current 0, to avoid some potential unexpected errors in codes since 0 is
> default value for integer variables.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
