[
https://issues.apache.org/jira/browse/HDFS-303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750046#action_12750046
]
Konstantin Shvachko commented on HDFS-303:
------------------------------------------
> 4. When renaming a file or directory to a non-existent directory (e.g. /a/b
> to /c/d, where /c doesn't exist) LocalFileSystem succeeds (returns true)
> while HDFS fails (false).
>
> HDFS seems more correct here.
Not sure. When HDFS creates a file under a non-existing path it calls
{{mkdirs()}} to create missing directories on the path. I think the rename
should follow the same pattern.
> 5. When renaming a file (or directory) as an existing file (or directory)
> LocalFileSystem succeeds (returns true) while HDFS fails (false).
The question is whether the behavior of LocalFileSystem is consistent across
different native local file systems. E.g. are the results the same on Windows
and ext3?
> The question here is do we allow file overwrites?
I don't think we should. POSIX rename() has very overloaded semantics. We
should keep it simple.
My proposal is to fail rename whenever the target file (or directory) exists.
Does HDFS need to be consistent with LocalFileSystem in this is another
question. I think it is OK that they are inconsistent, especially if
LocalFileSystem itself may not be consistent with the rename results on
different platforms.
Here is what Java documentation says about its rename:
http://java.sun.com/javase/6/docs/api/java/io/File.html#renameTo%28java.io.File%29
"Many aspects of the behavior of this method are inherently platform-dependent:
The rename operation might not be able to move a file from one filesystem to
another, it might not be atomic, and it might not succeed if a file with the
destination abstract pathname already exists."
> Make contracts of LocalFileSystem and DistributedFileSystem consistent
> ----------------------------------------------------------------------
>
> Key: HDFS-303
> URL: https://issues.apache.org/jira/browse/HDFS-303
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Tom White
> Attachments: hadoop-4114.patch, HDFS-303-common-test-case.patch
>
>
> There are a number of edge cases that the two file system implementations
> handle differently. In particular:
> * When trying to make a directory under an existing file, HDFS throws an
> IOException while LocalFileSystem doesn't.
> * The FileSytem#listStatus(Path) method returns null for a non-existent file
> on HDFS, while LocalFileSytem returns an empty FileStatus array.
> * When trying to rename a non-existent path, LocalFileSystem throws an
> IOException, while HDFS returns false.
> * When renaming a file or directory to a non-existent directory (e.g. /a/b to
> /c/d, where /c doesn't exist) LocalFileSystem succeeds (returns true) while
> HDFS fails (false).
> * When renaming a file (or directory) as an existing file (or directory)
> LocalFileSystem succeeds (returns true) while HDFS fails (false).
> We should document the expected behaviour for these cases in FileSystem's
> javadoc, and make sure all implementations conform to it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
