[
https://issues.apache.org/jira/browse/HADOOP-19350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17902128#comment-17902128
]
Steve Loughran commented on HADOOP-19350:
-----------------------------------------
HADOOP-18930 covered add a "fs.s3a.performance" flag which explicitly turns off
the safety check.
We expect the file create to work, because all HEAD/LIST operations are
disabled.
If ozone is actually failing here, this is a good thing as it means that even
without the S3 API calls. you still can't break the posix "no file above a
directory" semantics.
Not a bug in the test, more that some test stores have stricter semantics than
pure S3. The {{ITestS3AContractCreate}} is parameterized on the performance
flag, and expects to not see an error if the performance flag is true.
Someone will have to tune the test setup so that another flag in the
auth-keys.xml file can change the option of what to expect here on those
stores.
bq. See https://github.com/apache/hadoop/pull/6168#issuecomment-1756082091
That was us discovering the need to not expect errors when the performance flag
is true. Will need to be revisited by someone from the ozone dev group or
someone else with a store which is failing the test.
> ITestS3AContractCreate#testOverwrite*EmptyDirectory fail
> --------------------------------------------------------
>
> Key: HADOOP-19350
> URL: https://issues.apache.org/jira/browse/HADOOP-19350
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 3.4.1
> Reporter: Chung En Lee
> Priority: Minor
>
> See [https://github.com/apache/hadoop/pull/6168#issuecomment-1756082091]
> They also fail with Apache Ozone. See
> https://issues.apache.org/jira/browse/HDDS-11663
> Both of these tests expect to throw a FileAlreadyExistsException, while
> creating a file that overwrites an existing directory.
> The following is a step-by-step description of the test.
> # mkdir test/testOverwriteNonEmptyDirectory/ // existing directory
> # get test/testOverwriteNonEmptyDirector // start to create
> file to overwrite directory
> # check the file status of test/testOverwriteNonEmptyDirector
> ** If it throws FileNotFound then goes to step 4
> ** check if it is a directory then throw FileAlreadyExistsException
> // expect to throw
> [here|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L2196-L2199]
> ** check if the overwrite is false then throw FileAlreadyExistsException
> # put test/testOverwriteNonEmptyDirector
> However, in step 3 it actually goes to the first check, and put a new object.
> Because S3a is expecting:
> Get test/testOverwriteNonEmptyDirector // does not exist
> Get test/testOverwriteNonEmptyDirector/ // exists
> And it will still throw an exception but not FileAlreadyExistsException.
>
> How to fix:
> Maybe we also get the path test test/testOverwriteNonEmptyDirector/ and check
> if the directory exists.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
