[
https://issues.apache.org/jira/browse/HADOOP-9361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-9361:
-----------------------------------
Attachment: HADOOP-9361-006.patch
Patch with spec consistent with almost all HDFS behaviour, s3 and localfs set
up to throw tighter exceptions on some failures, and to throw EOFException on
seek(-negative).
h2. LocalFS behaviours to resolve
1. attempting to mkdir over an existing file returns false instead of raising
an exception.
propose: raise an exception. Nobody ever checks the return code from mkdirs
after all -so we should uprate it to be on a par with HDFS.
2. you can seek on an stream valid after a close()
options:
* fix
* ignore on the basis reads or writes will fail when attempted
3. if you rename a file over an existing file, the operation succeeds. This is
what bash does.
propose: document as the less preferred option; relax test to permit with a warn
h2. HDFS contract test behaviours
1. if you open a stream for append, rename the file and then do the append, the
old filename remains.
Propose: specify the outcome as "undefined"
2. if you attempt to rename a file that doesn't exist to a path in the same
directory, it returns false, rather than raising a FileNotFoundException
I'm assuming here that the dest path is being checked before the source. I'd
consider this an error
3. delete("/", true) returns false and doesn't delete anything
I've documented this as valid behaviour, and noted it is what HDFS does.
> Strictly define the expected behavior of filesystem APIs and write tests to
> verify compliance
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-9361
> URL: https://issues.apache.org/jira/browse/HADOOP-9361
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, test
> Affects Versions: 3.0.0, 2.2.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: HADOOP-9361-001.patch, HADOOP-9361-002.patch,
> HADOOP-9361-003.patch, HADOOP-9361-004.patch, HADOOP-9361-005.patch,
> HADOOP-9361-006.patch
>
>
> {{FileSystem}} and {{FileContract}} aren't tested rigorously enough -while
> HDFS gets tested downstream, other filesystems, such as blobstore bindings,
> don't.
> The only tests that are common are those of {{FileSystemContractTestBase}},
> which HADOOP-9258 shows is incomplete.
> I propose
> # writing more tests which clarify expected behavior
> # testing operations in the interface being in their own JUnit4 test classes,
> instead of one big test suite.
> # Having each FS declare via a properties file what behaviors they offer,
> such as atomic-rename, atomic-delete, umask, immediate-consistency -test
> methods can downgrade to skipped test cases if a feature is missing.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
