[
https://issues.apache.org/jira/browse/HDFS-10996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15965109#comment-15965109
]
Andrew Wang commented on HDFS-10996:
------------------------------------
Hi Sammi, thanks for working on this, sorry for the slow review:
bq. I go through the HdfsAdmin and find there is not a single create function
exposed there. So I'm not sure If I should add a new create API and will that
benefit its users? Uploaded v5 patch.
Sure, we can tackle this in a separate issue. We're doing some internal testing
and already found a few issues already regarding the lack of hflush support for
EC files (e.g. HDFS-11643, and apparently YARN job history too). These could
potentially be solved by having these apps specify an explicit "replicated"
policy, which requires a public API.
Some other code review nits, +1 pending these and Jenkins:
* getErasureCodingPolicyByName, we could assert hasReadLock() instead, since
this method doesn't do any writes
* Need a rebase since ECPolicies have moved to the new
SystemErasureCodingPolicies class
* testFileLevelECPolicy: nit: "policy should be found" -> "policy should not be
found"
> Ability to specify per-file EC policy at create time
> ----------------------------------------------------
>
> Key: HDFS-10996
> URL: https://issues.apache.org/jira/browse/HDFS-10996
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: erasure-coding
> Affects Versions: 3.0.0-alpha1
> Reporter: Andrew Wang
> Assignee: SammiChen
> Labels: hdfs-ec-3.0-nice-to-have
> Attachments: HDFS-10996-v1.patch, HDFS-10996-v2.patch,
> HDFS-10996-v3.patch, HDFS-10996-v4.patch, HDFS-10996-v5.patch
>
>
> Based on discussion in HDFS-10971, it would be useful to specify the EC
> policy when the file is created. This is useful for situations where app
> requirements do not map nicely to the current directory-level policies.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
