[
https://issues.apache.org/jira/browse/HADOOP-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272089#comment-15272089
]
Andras Bokor commented on HADOOP-13082:
---------------------------------------
[[email protected]], [~cnauroth],
Thank you guys for getting back to me.
First of all, yes, fallback logic was added way before HADOOP-9805.
I looked into it a bit deeper and it seems this rename situation is not that
simple. We have 9 methods in FileSystemContractBaseTest which is broken and 6
of 9 is broken by fallback logic.
I have a small list about it (it was my notes during the debugging):
{code}testRenameFileMoveToNonExistentDirectory
Fallback creates the missing directory
testRenameDirectoryAsExistingFile
Move fails but fallback behavior throws exception. Test expects only a
false
testRenameDirectoryMoveToNonExistentDirectory
Fallback creates the missing directory
testRenameChildDirForbidden
Fallback will throw an exception (cannot copy x to ist subdirectory y)
but contract expects just a false
testMoveDirUnderParent
renameTo return false (which is good) but fallback throws exception
while test does not expect exception
testRenameNonExistentPath
rename returns with false but fallback throws exception
{code}
bq. I'm not sure now, after all, the raw OS doesn't do this, does it?
Yes, it does. In case of one volume mv command do just a rename. The inode will
not change. In case of multi-volume the OS does a copy-delete.
I checked Jakarta IO commons and Guava how do they handle this and they use the
same solution as we do in RawLocalFileSystem (if renameTo false, do a
copy-delete). It is not good for us because we do not know why the renameTo
returned false (so we can't decide whether the false was returned due to
cross-volume move or other problem).
>From JDK 7 a new method was introduced.
>[Files.move|https://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#move%28java.nio.file.Path,%20java.nio.file.Path,%20java.nio.file.CopyOption...%29]
> do more or less the same as renameTo and it handles the cross-volume
>situation.
Now I am confused what to do:
# Skip this one and solve others
# Skip all the rename related contracts
# Solve all the broken contracts with FileUtil#copy
# Solve all the contracts with
[Files.move|https://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#move%28java.nio.file.Path,%20java.nio.file.Path,%20java.nio.file.CopyOption...%29]
# Other
What do you guys think?
> RawLocalFileSystem does not fail when moving file to a non-existing directory
> -----------------------------------------------------------------------------
>
> Key: HADOOP-13082
> URL: https://issues.apache.org/jira/browse/HADOOP-13082
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.0
> Reporter: Andras Bokor
> Assignee: Andras Bokor
>
> FileSystemContractBaseTest#testRenameFileMoveToNonExistentDirectory: creates
> a file then move it to a non-existing directory. It should fail but it will
> not (with RawLocalFileSystem) because in RawLocalFileSystem#rename(Path,
> Path) method we have a fallback behavior that accomplishes the rename by a
> full copy. The full copy will create the new directory and copy the file
> there.
> I see two possible solutions here:
> # Remove the fallback full copy behavior
> # Before full cp we should check whether the parent directory exists or not.
> If not return false an do not do the full copy.
> The fallback logic was added by
> [HADOOP-9805|https://issues.apache.org/jira/browse/HADOOP-9805].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
