[
https://issues.apache.org/jira/browse/HDFS-15596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17201336#comment-17201336
]
Uma Maheswara Rao G commented on HDFS-15596:
--------------------------------------------
distcp fails with the following error:
{code:java}
Error: java.io.IOException: File copy failed: hdfs://ns1/test/test.txt -->
hdfs://ns1/OzoneTest/test/test.txt at
org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:262)
at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:219) at
org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:48) at
org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at
org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168) Caused by:
java.io.IOException: Couldn't run retriable-command: Copying
hdfs://ns1/test/test.txt to hdfs://ns1/OzoneTest/test/test.txt at
org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:101)
at
org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:258)
... 10 more Caused by: java.lang.UnsupportedOperationException: This
API:create is specific to DFS. Can't run on other fs:o3fs://bucket.vol.ozone1
at
org.apache.hadoop.hdfs.ViewDistributedFileSystem.checkDFS(ViewDistributedFileSystem.java:402)
at
org.apache.hadoop.hdfs.ViewDistributedFileSystem.create(ViewDistributedFileSystem.java:391)
at
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.copyToFile(RetriableFileCopyCommand.java:201)
at
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:143)
at
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:115)
at
org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87)
... 11 more{code}
> ViewHDFS#create(f, permission, cflags, bufferSize, replication, blockSize,
> progress, checksumOpt) should not be restricted to DFS only.
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-15596
> URL: https://issues.apache.org/jira/browse/HDFS-15596
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Uma Maheswara Rao G
> Assignee: Uma Maheswara Rao G
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The ViewHDFS#create(f, permission, cflags, bufferSize, replication,
> blockSize, progress, checksumOpt) API already available in FileSystem. It
> will use other overloaded API and finally can go to ViewFileSystem. This case
> works in regular ViewFileSystem also. With ViewHDFS, we restricted this to
> DFS only which cause discp to fail when target is non hdfs as it's using this
> API.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
