[
https://issues.apache.org/jira/browse/HADOOP-17742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jerry updated HADOOP-17742:
---------------------------
Description:
{code:java}
When I copy file from a ftp filesystem to other ftp filesystem by distCp, the
tmp target path can't be found. After my test, it was caused by the comma. And
when the uri of target path with schema, host, port can't be rename because of
the ioexception.
// exception information
java.io.FileNotFoundException: Source path
ftp://hadoop336.photo.163.org:21/ndi-0111/56598489-3de3-4585-8426-c03435640481/.distcp.tmp.attempt_local344388648_0001_m_000000_0.1622634094600
does not existjava.io.FileNotFoundException: Source path
ftp://hadoop336.photo.163.org:21/ndi-0111/56598489-3de3-4585-8426-c03435640481/.distcp.tmp.attempt_local344388648_0001_m_000000_0.1622634094600
does not exist at
org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:668) at
org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:626) at
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.promoteTmpToTarget(RetriableFileCopyCommand.java:220)
at
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:155)
at
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:115)
at
org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87)
at
org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:258)
// exception information when rename
Exception in thread "main" java.io.IOException: Cannot rename source:
/ndi-0111/2/distcp.tmp.attempt_local1899244355_0001_m_000000_0.1622622033323 to
ftp://59.111.93.226:60021/ndi-0111/2/test6.txt -only same directory renames are
supportedException in thread "main" java.io.IOException: Cannot rename source:
/ndi-0111/2/distcp.tmp.attempt_local1899244355_0001_m_000000_0.1622622033323 to
ftp://59.111.93.226:60021/ndi-0111/2/test6.txt -only same directory renames are
supported at
org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:674) at
org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:613)
{code}
was:
{code:java}
When I copy file from a ftp filesystem to other ftp filesystem, the tmp target
path can't be found. After my test, it was caused by the comma. And when the
uri of target path with schema, host, port can't be rename.
// exception information
java.io.FileNotFoundException: Source path
ftp://hadoop336.photo.163.org:21/ndi-0111/56598489-3de3-4585-8426-c03435640481/.distcp.tmp.attempt_local344388648_0001_m_000000_0.1622634094600
does not existjava.io.FileNotFoundException: Source path
ftp://hadoop336.photo.163.org:21/ndi-0111/56598489-3de3-4585-8426-c03435640481/.distcp.tmp.attempt_local344388648_0001_m_000000_0.1622634094600
does not exist at
org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:668) at
org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:626) at
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.promoteTmpToTarget(RetriableFileCopyCommand.java:220)
at
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:155)
at
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:115)
at
org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87)
at
org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:258)
{code}
> DistCp: distcp fail when copying to ftp filesystem
> --------------------------------------------------
>
> Key: HADOOP-17742
> URL: https://issues.apache.org/jira/browse/HADOOP-17742
> Project: Hadoop Common
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 3.3.0
> Reporter: jerry
> Assignee: jerry
> Priority: Blocker
>
> {code:java}
> When I copy file from a ftp filesystem to other ftp filesystem by distCp, the
> tmp target path can't be found. After my test, it was caused by the comma.
> And when the uri of target path with schema, host, port can't be rename
> because of the ioexception.
> // exception information
> java.io.FileNotFoundException: Source path
> ftp://hadoop336.photo.163.org:21/ndi-0111/56598489-3de3-4585-8426-c03435640481/.distcp.tmp.attempt_local344388648_0001_m_000000_0.1622634094600
> does not existjava.io.FileNotFoundException: Source path
> ftp://hadoop336.photo.163.org:21/ndi-0111/56598489-3de3-4585-8426-c03435640481/.distcp.tmp.attempt_local344388648_0001_m_000000_0.1622634094600
> does not exist at
> org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:668) at
> org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:626) at
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.promoteTmpToTarget(RetriableFileCopyCommand.java:220)
> at
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:155)
> at
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:115)
> at
> org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87)
> at
> org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:258)
> // exception information when rename
> Exception in thread "main" java.io.IOException: Cannot rename source:
> /ndi-0111/2/distcp.tmp.attempt_local1899244355_0001_m_000000_0.1622622033323
> to ftp://59.111.93.226:60021/ndi-0111/2/test6.txt -only same directory
> renames are supportedException in thread "main" java.io.IOException: Cannot
> rename source:
> /ndi-0111/2/distcp.tmp.attempt_local1899244355_0001_m_000000_0.1622622033323
> to ftp://59.111.93.226:60021/ndi-0111/2/test6.txt -only same directory
> renames are supported at
> org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:674) at
> org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:613)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
