[
https://issues.apache.org/jira/browse/HDFS-303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tom White updated HDFS-303:
---------------------------
Attachment: HDFS-303-common-test-case.patch
Following HDFS-538, it would be good to remove these inconsistencies. Here is
what I think we should do for each of them (the attached test shows how
LocalFileSystem fails compared to HDFS). I'd be grateful to get others thoughts
on this.
bq. 1. When trying to make a directory under an existing file, HDFS throws an
IOException while LocalFileSystem doesn't.
An IOException seems good here.
bq. 2. The FileSytem#listStatus(Path) method returns null for a non-existent
file on HDFS, while LocalFileSytem returns an empty FileStatus array.
Fixed in HDFS-538
bq. 3. When trying to rename a non-existent path, LocalFileSystem throws an
FileNotFoundException, while HDFS returns false.
FileNotFoundException seems appropriate here.
bq. 4. When renaming a file or directory to a non-existent directory (e.g. /a/b
to /c/d, where /c doesn't exist) LocalFileSystem succeeds (returns true) while
HDFS fails (false).
HDFS seems more correct here.
bq. 5. When renaming a file (or directory) as an existing file (or directory)
LocalFileSystem succeeds (returns true) while HDFS fails (false).
There are two cases:
i. Rename file to existing file
The question here is do we allow file overwrites? LocalFileSystem and POSIX
rename (http://www.opengroup.org/onlinepubs/009695399/functions/rename.html)
both do, but HDFS historically has not. Is there a reason not to allow this?
ii. Rename directory to existing directory
If the existing directory is empty then it is reasonable for the rename to
succeed. Otherwise it should fail. (This is consistent with POSIX)
Another problem is that rename() and mkdirs() may either return false or throw
an IOException to indicate failure. Clients need to check both cases to be sure
if the operation succeeded. Ideally we would just use one mechanism
(exceptions), but that is not so easy to migrate to since we can't overload on
return type (boolean and void).
Also, LocalFileSystem delegates rename() to java.io.File#renameTo(), which is
platform-dependent. For this reason some of these corner cases should be
explicitly handled by LocalFileSystem itself.
> Make contracts of LocalFileSystem and DistributedFileSystem consistent
> ----------------------------------------------------------------------
>
> Key: HDFS-303
> URL: https://issues.apache.org/jira/browse/HDFS-303
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Tom White
> Attachments: hadoop-4114.patch, HDFS-303-common-test-case.patch
>
>
> There are a number of edge cases that the two file system implementations
> handle differently. In particular:
> * When trying to make a directory under an existing file, HDFS throws an
> IOException while LocalFileSystem doesn't.
> * The FileSytem#listStatus(Path) method returns null for a non-existent file
> on HDFS, while LocalFileSytem returns an empty FileStatus array.
> * When trying to rename a non-existent path, LocalFileSystem throws an
> IOException, while HDFS returns false.
> * When renaming a file or directory to a non-existent directory (e.g. /a/b to
> /c/d, where /c doesn't exist) LocalFileSystem succeeds (returns true) while
> HDFS fails (false).
> * When renaming a file (or directory) as an existing file (or directory)
> LocalFileSystem succeeds (returns true) while HDFS fails (false).
> We should document the expected behaviour for these cases in FileSystem's
> javadoc, and make sure all implementations conform to it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
