[
https://issues.apache.org/jira/browse/HDFS-15592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uma Maheswara Rao G updated HDFS-15592:
---------------------------------------
Summary: DistCP fails with ViewHDFS and preserveEC options if the actual
target path is non HDFS (was: DistCP fails with ViewHDFS if the actual target
path is non HDFS)
> DistCP fails with ViewHDFS and preserveEC options if the actual target path
> is non HDFS
> ---------------------------------------------------------------------------------------
>
> Key: HDFS-15592
> URL: https://issues.apache.org/jira/browse/HDFS-15592
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: viewfs, ViewHDFS
> Affects Versions: 3.4.0
> Reporter: Uma Maheswara Rao G
> Assignee: Uma Maheswara Rao G
> Priority: Major
>
> When we configure target path mount point with Ozone (or any other fs),
> distcp will fail.
> The reason is, if the src path having ec policy enabled, it will try to
> retain that properties.SO, in this case it is using DFS specific createFile
> API.
> But here we have to ensure, tareget path can from non hdfs in ViewHDFS case.
> In RetriayableFIleCopyCommand#copyToFile, we should fix the following piece
> of code.
>
> {code:java}
> if (preserveEC && sourceStatus.isErasureCoded()
> && sourceStatus instanceof HdfsFileStatus
> && targetFS instanceof DistributedFileSystem) {
> ecPolicy = ((HdfsFileStatus) sourceStatus).getErasureCodingPolicy();
> }{code}
>
> Here it's just checking targetFs instanceof DistributedFileSystem, but in
> ViewHDFS case, fs will be DFS only but actual target can point to mounted fs.
> So, to handle this case, we should use resolvePath API and check the resolved
> target path scheme is dfs or or not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
