[
https://issues.apache.org/jira/browse/HADOOP-16878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17069312#comment-17069312
]
Ayush Saxena edited comment on HADOOP-16878 at 3/28/20, 8:36 AM:
-----------------------------------------------------------------
Thanx [~gabor.bota] for the report and PR.
I tried this with a quick UT in HDFS. Seems I didn't get the exception thrown :
Sharing the UT :
{code:java}
@Test
public void testCopyBetweenFsEqualPath() throws Exception {
Configuration conf = getTestConfiguration();
try (MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).build()) {
cluster.waitActive();
final DistributedFileSystem dfs = cluster.getFileSystem();
Path filePath = new Path("/dir/file");
dfs.create(filePath).close();
FileStatus fstatus = dfs.getFileStatus(filePath);
LambdaTestUtils.intercept(PathOperationException.class,
() -> FileUtil.copy(dfs, fstatus, dfs, filePath, false, true, conf));
}
}
{code}
(Can place this in TestDistributedFileSystem.java)
The reason being :
{code:java}
if (src.toUri().equals(dst.toUri())) {
{code}
In {{src}} the scheme is present but in {{dst}} it isn't present.
Can you give a check once.
was (Author: ayushtkn):
Thanx [~gabor.bota] for the report and PR.
I tried this with a quick UT in HDFS. Seems I didn't get the exception thrown :
Sharing the UT :
{code:java}
@Test
public void testCopyBetweenFsEqualPath() throws Exception {
Configuration conf = getTestConfiguration();
try (MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).build()) {
cluster.waitActive();
final DistributedFileSystem dfs = cluster.getFileSystem();
Path filePath = new Path("/dir/file");
dfs.create(filePath).close();
FileStatus fstatus = dfs.getFileStatus(filePath);
LambdaTestUtils.intercept(PathOperationException.class,
() -> FileUtil.copy(dfs, fstatus, dfs, filePath, false, true, conf));
}
}
{code}
(Can place in this TestDistributedFileSystem.java)
The reason being :
{code:java}
if (src.toUri().equals(dst.toUri())) {
{code}
In {{src}} the scheme is present but in {{dst}} it isn't present.
Can you give a check once.
> Copy command in FileUtil to throw an exception if the source and destination
> is the same
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-16878
> URL: https://issues.apache.org/jira/browse/HADOOP-16878
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 3.3.0
> Reporter: Gabor Bota
> Assignee: Gabor Bota
> Priority: Major
>
> We encountered an error during a test in our QE when the file destination and
> source path were the same. This happened during an ADLS test, and there were
> no meaningful error messages, so it was hard to find the root cause of the
> failure.
> The error we saw was that file size has changed during the copy operation.
> The new file creation in the destination - which is the same as the source -
> creates a file and sets the file length to zero. After this, getting the
> source file will fail because the sile size changed during the operation.
> I propose a solution to at least log in error level in the {{FileUtil}} if
> the source and destination of the copy operation is the same, so debugging
> issues like this will be easier in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
