[
https://issues.apache.org/jira/browse/HADOOP-9361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14029976#comment-14029976
]
Andrew Wang commented on HADOOP-9361:
-------------------------------------
Hi Steve,
I'm going to take it on faith that you applied all the doc improvements, since
honestly there's too much too look through :) Going to focus on code questions
this time around. It's still hard for me to run the S3 / Swift tests since I'd
have to get credentials, but I can do it if needed.
* Only big feedback I have, is there some way we could have a single parent
test for each FS that runs all its contracts? All the concrete
TestFooBarContract classes are basically copy paste, I'd like to get rid of
them. It's also harder for poor me to run all the HDFS contract tests.
* site.xml, link is still not quite right, should be filesystem/index.html
* FSExceptionMessages, should we re-use the Java messages for these exceptions?
I think they're a bit different. There are also a bunch of places where these
strings *aren't* used, but unifying it everywhere is a tall order. Maybe a
follow-on JIRA. Another maybe nice follow-on would be having a factory method
for things like FileAlreadyExistsException where we stitch something to a
string.
* I'd prefer to flip "fs.contract.supports-seek-past-eof" so we can have all
trues for HDFS's contract. Strawman, "supports-exception-on-seek-past-eof"?
* Jets3tnative: typo: maxListingLengthm
* Jets3tnative: there are two handleBlahException methods that just delegate to
handleException. Can we just get rid of them?
* NativeS3, read's LOG.info has an extra "{}"
* Some Contract class javadocs are inconsistent. HDFSContract's class javadoc
says "Local filesystem". SwiftContract mentions S3N.
I think we're pretty close, modulo the big one. Thanks again for the effort
here.
> Strictly define the expected behavior of filesystem APIs and write tests to
> verify compliance
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-9361
> URL: https://issues.apache.org/jira/browse/HADOOP-9361
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, test
> Affects Versions: 3.0.0, 2.4.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: HADOOP-9361-001.patch, HADOOP-9361-002.patch,
> HADOOP-9361-003.patch, HADOOP-9361-004.patch, HADOOP-9361-005.patch,
> HADOOP-9361-006.patch, HADOOP-9361-007.patch, HADOOP-9361-008.patch,
> HADOOP-9361-009.patch, HADOOP-9361-011.patch, HADOOP-9361-012.patch,
> HADOOP-9361-013.patch, HADOOP-9361-014.patch, HADOOP-9361-015.patch,
> HADOOP-9361.awang-addendum.patch
>
>
> {{FileSystem}} and {{FileContract}} aren't tested rigorously enough -while
> HDFS gets tested downstream, other filesystems, such as blobstore bindings,
> don't.
> The only tests that are common are those of {{FileSystemContractTestBase}},
> which HADOOP-9258 shows is incomplete.
> I propose
> # writing more tests which clarify expected behavior
> # testing operations in the interface being in their own JUnit4 test classes,
> instead of one big test suite.
> # Having each FS declare via a properties file what behaviors they offer,
> such as atomic-rename, atomic-delete, umask, immediate-consistency -test
> methods can downgrade to skipped test cases if a feature is missing.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
