[
https://issues.apache.org/jira/browse/HADOOP-16878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-16878:
------------------------------------
Description:
We encountered an error during a test in our QE when the file destination and
source path were the same. This happened during an ADLS test, and there were no
meaningful error messages, so it was hard to find the root cause of the failure.
The error we saw was that file size has changed during the copy operation. The
new file creation in the destination - which is the same as the source -
creates a file and sets the file length to zero. After this, getting the source
file will fail because the sile size changed during the operation.
I propose a solution to at least log in error level in the {{FileUtil}} if the
source and destination of the copy operation is the same, so debugging issues
like this will be easier in the future.
Update: This has been reverted as it caused problems (
HADOOP-17936
was:
We encountered an error during a test in our QE when the file destination and
source path were the same. This happened during an ADLS test, and there were no
meaningful error messages, so it was hard to find the root cause of the failure.
The error we saw was that file size has changed during the copy operation. The
new file creation in the destination - which is the same as the source -
creates a file and sets the file length to zero. After this, getting the source
file will fail because the sile size changed during the operation.
I propose a solution to at least log in error level in the {{FileUtil}} if the
source and destination of the copy operation is the same, so debugging issues
like this will be easier in the future.
Update: This has been reverted as it caused
> FileUtil.copy() to throw IOException if the source and destination are the
> same
> -------------------------------------------------------------------------------
>
> Key: HADOOP-16878
> URL: https://issues.apache.org/jira/browse/HADOOP-16878
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 3.3.0
> Reporter: Gabor Bota
> Assignee: Gabor Bota
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: hdfsTest.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> We encountered an error during a test in our QE when the file destination and
> source path were the same. This happened during an ADLS test, and there were
> no meaningful error messages, so it was hard to find the root cause of the
> failure.
> The error we saw was that file size has changed during the copy operation.
> The new file creation in the destination - which is the same as the source -
> creates a file and sets the file length to zero. After this, getting the
> source file will fail because the sile size changed during the operation.
> I propose a solution to at least log in error level in the {{FileUtil}} if
> the source and destination of the copy operation is the same, so debugging
> issues like this will be easier in the future.
> Update: This has been reverted as it caused problems (
> HADOOP-17936
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
