[
https://issues.apache.org/jira/browse/HADOOP-17742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jerry updated HADOOP-17742:
---------------------------
Description:
{code:java}
When I copy file from a ftp filesystem to other ftp filesystem by distCp, there
are two question. Firstly, the tmp target path can't be found in the exception
information. After my test, it was caused by the comma. Then when the uri of
target path with schema, host, port can't be rename because of the ioexception.
// exception information
java.io.FileNotFoundException: Source path
ftp://hadoop336.photo.163.org:21/ndi-0111/56598489-3de3-4585-8426-c03435640481/.distcp.tmp.attempt_local344388648_0001_m_000000_0.1622634094600
does not existjava.io.FileNotFoundException: Source path
ftp://hadoop336.photo.163.org:21/ndi-0111/56598489-3de3-4585-8426-c03435640481/.distcp.tmp.attempt_local344388648_0001_m_000000_0.1622634094600
does not exist at
org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:668) at
org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:626) at
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.promoteTmpToTarget(RetriableFileCopyCommand.java:220)
at
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:155)
at
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:115)
at
org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87)
at
org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:258)
{code}
When I update the tempFileName from '.distcp.tmp.XXX' in
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand#getTempFile to
'.distcp.tmp.XXX', the question one is gone.
was:
{code:java}
When I copy file from a ftp filesystem to other ftp filesystem by distCp, the
tmp target path can't be found in the exception information. After my test, it
was caused by the comma. And when the uri of target path with schema, host,
port can't be rename because of the ioexception.
// exception information
java.io.FileNotFoundException: Source path
ftp://hadoop336.photo.163.org:21/ndi-0111/56598489-3de3-4585-8426-c03435640481/.distcp.tmp.attempt_local344388648_0001_m_000000_0.1622634094600
does not existjava.io.FileNotFoundException: Source path
ftp://hadoop336.photo.163.org:21/ndi-0111/56598489-3de3-4585-8426-c03435640481/.distcp.tmp.attempt_local344388648_0001_m_000000_0.1622634094600
does not exist at
org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:668) at
org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:626) at
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.promoteTmpToTarget(RetriableFileCopyCommand.java:220)
at
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:155)
at
org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:115)
at
org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87)
at
org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:258)
{code}
> DistCp: distcp fail when copying to ftp filesystem
> --------------------------------------------------
>
> Key: HADOOP-17742
> URL: https://issues.apache.org/jira/browse/HADOOP-17742
> Project: Hadoop Common
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 3.3.0
> Reporter: jerry
> Assignee: jerry
> Priority: Blocker
>
> {code:java}
> When I copy file from a ftp filesystem to other ftp filesystem by distCp,
> there are two question. Firstly, the tmp target path can't be found in the
> exception information. After my test, it was caused by the comma. Then when
> the uri of target path with schema, host, port can't be rename because of the
> ioexception.
> // exception information
> java.io.FileNotFoundException: Source path
> ftp://hadoop336.photo.163.org:21/ndi-0111/56598489-3de3-4585-8426-c03435640481/.distcp.tmp.attempt_local344388648_0001_m_000000_0.1622634094600
> does not existjava.io.FileNotFoundException: Source path
> ftp://hadoop336.photo.163.org:21/ndi-0111/56598489-3de3-4585-8426-c03435640481/.distcp.tmp.attempt_local344388648_0001_m_000000_0.1622634094600
> does not exist at
> org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:668) at
> org.apache.hadoop.fs.ftp.FTPFileSystem.rename(FTPFileSystem.java:626) at
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.promoteTmpToTarget(RetriableFileCopyCommand.java:220)
> at
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:155)
> at
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:115)
> at
> org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87)
> at
> org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:258)
> {code}
> When I update the tempFileName from '.distcp.tmp.XXX' in
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand#getTempFile to
> '.distcp.tmp.XXX', the question one is gone.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
