[
https://issues.apache.org/jira/browse/HDFS-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Walter Su updated HDFS-8854:
----------------------------
Attachment: HDFS-8854.00.patch
bq. Maybe we can keep ECSchema class in COMMON, and refactor HDFS codes to use
ErasureCodingPolicy instead?
Yeah, that's better. I updated the jira description.
Upload initial patch. I'll revisit the indent and javadoc later.
The patch does:
1. add {{ECPolicy}} in hadoop-hdfs. {{ECSchema}} is left and only used in
hadoop-common(codec).
2. Rename {{SchemaLoader}} to {{ECPolicyLoader}},
{{ErasureCodingSchemaManager}} to {{ErasureCodingPolicyManager}}.
3. {{ECPolicy}} is loaded from file. The file definition is :
{code}
<policies>
<policy>
<name>RS-6-3-64k</name>
<schema>
<numDataUnits>6</numDataUnits>
<numParityUnits>3</numParityUnits>
<codec>RS</codec>
</schema>
<cellSize>65536</cellSize>
</policy>
<policy>
<name>RS-10-4-128k</name>
<schema>
<numDataUnits>10</numDataUnits>
<numParityUnits>4</numParityUnits>
<codec>RS</codec>
</schema>
<cellSize>131072</cellSize>
</policy>
</policies>
{code}
4. {{ECSchema}} is not loaded from file anymore. {{ECSchema}} *is got from*
{{ECPolicy}}.
5. {{ECPolicy}} is cached instead of {{ECSchema}}.
6. update CLI. cellSize is no longer accepted.
7. pass around {{ECPolicy}} instead of {{ECSchema}} + {{cellSize}}. Update
proto, protocol.
hints for review: The patch is huge because of #7. You can focus on #1~#6.
> Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs
> ----------------------------------------------------------------------
>
> Key: HDFS-8854
> URL: https://issues.apache.org/jira/browse/HDFS-8854
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Walter Su
> Assignee: Walter Su
> Attachments: HDFS-8854.00.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
