[
https://issues.apache.org/jira/browse/HDFS-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy updated HDFS-10971:
--------------------------------------
Attachment: HDFS-10971.02.patch
Thanks for the review [~jojochuang] and [~andrew.wang]. Attaching v02 patch
with following comments addressed. Please take a look.
Code comments fixed as per the suggestion.
bq. It'd be good to add messages to the asserts as a form of documentation.
Added
bq. The test is also named "testPreserve..." whereas we might want to name it
"testReplFactorNotPreserved..." or "...Ignored..." for clarity
Done. Test renamed to testReplFactorNotPreservedOnErasureCodedFile
bq. Consider doing a static import on the asserts to make them a little more
concise
Done
bq. We test EC src and repl dest, should we also test repl src and EC dst?
Also add EC to EC test with different EC policies for completeness?
Done
> Distcp should not copy replication factor if source file is erasure coded
> -------------------------------------------------------------------------
>
> Key: HDFS-10971
> URL: https://issues.apache.org/jira/browse/HDFS-10971
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: distcp
> Affects Versions: 3.0.0-alpha1
> Reporter: Wei-Chiu Chuang
> Assignee: Manoj Govindassamy
> Priority: Blocker
> Labels: hdfs-ec-3.0-must-do
> Attachments: HDFS-10971.01.patch, HDFS-10971.02.patch,
> HDFS-10971.testcase.patch
>
>
> The current erasure coding implementation uses replication factor field to
> store erasure coding policy.
> Distcp copies the source file's replication factor to the destination if
> {{-pr}} is specified. However, if the source file is EC, the replication
> factor (which is EC policy) should not be replicated to the destination file.
> When a HdfsFileStatus is converted to FileStatus, the replication factor is
> set to 0 if it's an EC file.
> In fact, I will attach a test case that shows trying to replicate the
> replication factor of an EC file results in an IOException: "Requested
> replication factor of 0 is less than the required minimum of 1 for
> /tmp/dst/dest2"
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
