Uma Maheswara Rao G created HDFS-15592:
------------------------------------------
Summary: DistCP fails with ViewHDFS if the actual target path is
non HDFS
Key: HDFS-15592
URL: https://issues.apache.org/jira/browse/HDFS-15592
Project: Hadoop HDFS
Issue Type: Sub-task
Components: ViewHDFS, viewfs
Affects Versions: 3.4.0
Reporter: Uma Maheswara Rao G
Assignee: Uma Maheswara Rao G
When we configure target path mount point with Ozone (or any other fs), distcp
will fail.
The reason is, if the src path having ec policy enabled, it will try to retain
that properties.SO, in this case it is using DFS specific createFile API.
But here we have to ensure, tareget path can from non hdfs in ViewHDFS case.
In RetriayableFIleCopyCommand#copyToFile, we should fix the following piece of
code.
{code:java}
if (preserveEC && sourceStatus.isErasureCoded()
&& sourceStatus instanceof HdfsFileStatus
&& targetFS instanceof DistributedFileSystem) {
ecPolicy = ((HdfsFileStatus) sourceStatus).getErasureCodingPolicy();
}{code}
Here it's just checking targetFs instanceof DistributedFileSystem, but in
ViewHDFS case, fs will be DFS only but actual target can point to mounted fs.
So, to handle this case, we should use resolvePath API and check the resolved
target path scheme is dfs or or not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
