[ https://issues.apache.org/jira/browse/HDFS-14190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16947850#comment-16947850 ]
Wei-Chiu Chuang commented on HDFS-14190: ---------------------------------------- I suspect HDFS-14323 or HDFS-14423 fixed it. > Copying folders containing = - characters between hdfs (using webhdfs) does > not work in distcp > ---------------------------------------------------------------------------------------------- > > Key: HDFS-14190 > URL: https://issues.apache.org/jira/browse/HDFS-14190 > Project: Hadoop HDFS > Issue Type: Bug > Components: distcp > Affects Versions: 3.1.1 > Reporter: yinsong > Assignee: Aihua Xu > Priority: Major > > Copying folders containing = - characters between hdfs (using webhdfs) does > not work in distcp > for example: > src:hadoop2.7 target:hadoop3.1.1 > (1) > hadoop distcp \ > -pugp \ > -i \ > webhdfs://1.1.1.1:50070/sudiyi_datawarehouse > webhdfs://2.2.2.2:50070/sudiyi_datawarehouse > ERROR tools.SimpleCopyListing: FileNotFoundException exception in listStatus: > File /sudiyi_datawarehouse/st_device_standard_ds/date_time%3D2018-10-10 does > not exist > > (2) > hadoop distcp \ > -Dmapreduce.framework.name=yarn \ > -pugp \ > -i \ > webhdfs://1.1.1.1:50070/druid webhdfs://2.2.2.2:50070/druid > Error: java.io.IOException: File copy failed: > webhdfs://10.26.93.65:50070/druid/indexing-logs/kill_task-myapp_V1-2018-04-26T16_20_55+0800 > --> > webhdfs://10.27.234.198:50070/druid/indexing-logs/kill_task-myapp_V1-2018-04-26T16_20_55+0800 > at > org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:259) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:217) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:48) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168) > Caused by: java.io.IOException: Couldn't run retriable-command: Copying > webhdfs://10.26.93.65:50070/druid/indexing-logs/kill_task-myapp_V1-2018-04-26T16_20_55+0800 > to > webhdfs://10.27.234.198:50070/druid/indexing-logs/kill_task-myapp_V1-2018-04-26T16_20_55+0800 > at > org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:101) > at > org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:256) > ... 10 more > Caused by: java.io.IOException: Failed to promote > tmp-file:webhdfs://10.27.234.198:50070/druid/.distcp.tmp.attempt_1545990837043_0016_m_000015_2 > to: > webhdfs://10.27.234.198:50070/druid/indexing-logs/kill_task-myapp_V1-2018-04-26T16_20_55+0800 > at > org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.promoteTmpToTarget(RetriableFileCopyCommand.java:250) > at > org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:140) > at > org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:99) > at > org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87) -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org