[
https://issues.apache.org/jira/browse/HADOOP-19072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17872965#comment-17872965
]
ASF GitHub Bot commented on HADOOP-19072:
-----------------------------------------
steveloughran commented on PR #6543:
URL: https://github.com/apache/hadoop/pull/6543#issuecomment-2284312413
aah, even after the addendum i"m getting failures.
Viraj. copy my setting then rerun everything with -Dscale. Then also a run
with -Dprefix
[INFO]
[ERROR] Failures:
[ERROR]
ITestS3AFSMainOperations>FSMainOperationsBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:241
Should throw IOException.
[ERROR]
ITestS3AFileSystemContract>FileSystemContractBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:224
Should throw IOException.
[ERROR]
ITestS3AFileContextMainOperations>FileContextMainOperationsBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:276
Should throw IOException.
[ERROR]
ITestS3AFileContextURI>FileContextURIBase.testMkdirsFailsForSubdirectoryOfExistingFile:258
Should throw IOException.
[INFO]
```
```
[ERROR]
testMkdirsFailsForSubdirectoryOfExistingFile(org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextURI)
Time elapsed: 1.217 s <<< FAILURE!
java.lang.AssertionError: Should throw IOException.
at org.junit.Assert.fail(Assert.java:89)
at
org.apache.hadoop.fs.FileContextURIBase.testMkdirsFailsForSubdirectoryOfExistingFile(FileContextURIBase.java:258)
[ERROR]
testMkdirsFailsForSubdirectoryOfExistingFile(org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextMainOperations)
Time elapsed: 1.264 s <<< FAILURE!
java.lang.AssertionError: Should throw IOException.
at org.junit.Assert.fail(Assert.java:89)
at
org.apache.hadoop.fs.FileContextMainOperationsBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile(FileContextMainOperationsBaseTest.java:276)
[ERROR]
testMkdirsFailsForSubdirectoryOfExistingFile(org.apache.hadoop.fs.s3a.ITestS3AFileSystemContract)
Time elapsed: 0.856 s <<< FAILURE!
java.lang.AssertionError: Should throw IOException.
at org.junit.Assert.fail(Assert.java:89)
at
org.apache.hadoop.fs.FileSystemContractBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile(FileSystemContractBaseTest.java:224)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
[ERROR]
testMkdirsFailsForSubdirectoryOfExistingFile(org.apache.hadoop.fs.s3a.ITestS3AFSMainOperations)
Time elapsed: 0.844 s <<< FAILURE!
java.lang.AssertionError: Should throw IOException.
at org.junit.Assert.fail(Assert.java:89)
at
org.apache.hadoop.fs.FSMainOperationsBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile(FSMainOperationsBaseTest.java:241)
```
suspect I didn't do a full 3.4 test run before pushing it up on that branch,
instead just the modified files. my mistake. there'll be another patch to
backport
> S3A: expand optimisations on stores with "fs.s3a.performance.flags" for mkdir
> -----------------------------------------------------------------------------
>
> Key: HADOOP-19072
> URL: https://issues.apache.org/jira/browse/HADOOP-19072
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Assignee: Viraj Jasani
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.5.0
>
>
> on an s3a store with fs.s3a.create.performance set, speed up other operations
> * mkdir to skip parent directory check: just do a HEAD to see if there's a
> file at the target location
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
