[
https://issues.apache.org/jira/browse/HADOOP-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271503#comment-15271503
]
Chris Nauroth commented on HADOOP-13082:
----------------------------------------
HADOOP-9805 is mostly just a refactoring of a patch I wrote for HADOOP-9507 to
provide POSIX-like behavior on Windows by allowing full replacement of a
destination directory if it's empty. I think the only logical difference of
HADOOP-9805 is that it added an explicit check for running on Windows. I'd be
surprised if HADOOP-9805 introduced the problem.
I don't think either HADOOP-9507 or HADOOP-9805 are the root cause of the
current problem though. Instead, I think the behavior you see is a result of
the final {{FileUtil#copy}} call. That line of code goes waaaay back. (I
stopped digging through the git log at 2008, but the code even pre-dates that.)
We have to assume that there are wide ecosystem dependencies on this behavior
for {{RawLocalFileSystem}}, so I don't think it's realistic to change that
behavior right now. This is different from HDFS semantics, where the rename
would fail. The contract tests are largely derived from HDFS semantics. The
local file system is an unusual one-off in several ways, and clients have
different expectations for its semantics compared to HDFS or an alternative
Hadoop-compatible file system.
To address this test failure, I think it would be appropriate for the test
suite subclass to override {{testRenameFileMoveToNonExistentDirectory}} and
annotate it with {{@Ignore}}. The semantics exercised by the test are not
applicable to the local file system, so it makes sense to skip it.
Alternatively, we could introduce another contract flag to control the test's
behavior. I think that would be overkill though, because I'm pretty sure this
is going to be a one-off for the local file system.
> RawLocalFileSystem does not fail when moving file to a non-existing directory
> -----------------------------------------------------------------------------
>
> Key: HADOOP-13082
> URL: https://issues.apache.org/jira/browse/HADOOP-13082
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.0
> Reporter: Andras Bokor
> Assignee: Andras Bokor
>
> FileSystemContractBaseTest#testRenameFileMoveToNonExistentDirectory: creates
> a file then move it to a non-existing directory. It should fail but it will
> not (with RawLocalFileSystem) because in RawLocalFileSystem#rename(Path,
> Path) method we have a fallback behavior that accomplishes the rename by a
> full copy. The full copy will create the new directory and copy the file
> there.
> I see two possible solutions here:
> # Remove the fallback full copy behavior
> # Before full cp we should check whether the parent directory exists or not.
> If not return false an do not do the full copy.
> The fallback logic was added by
> [HADOOP-9805|https://issues.apache.org/jira/browse/HADOOP-9805].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
