steveloughran commented on PR #7329:
URL: https://github.com/apache/hadoop/pull/7329#issuecomment-2698513979
@saikatroy038
```
Another iteration;
* pulled out *all* the stuff related to the notion of a configuration
service -needs to be independent PR
* "simplified" logic as to when to use conditional create for a normal
create() call, even without any explicit request.
This comes down to: if you are using the performance flag for creation, or
when you create an fs with `create` in `fs.s3a.performance.flags` then we
switch to using condition create if:
overwrite=false and conditional create is available and the create is not a
conditional etag creation.
---
Somehow those conditional options from the builder of S3ABlockOutput stream
aren't getting in.
I actually realise these are duplicate; putting them into the
`PutObjectOptions` settings is enough
in S3AFileSystem, L2181, the PutObjectOptions created must set the
etag/conditional create flag
from that calculated in the `innerCreateFile`
## Tests.
### Performance flag && !overwrite triggers conditional create
Create a file with
```
opt("fs.s3a.create.performance",true)
.overwrite(false)
```
and verify that this is mapped to a conditional create
### Behaviour when FS doesn't support conditional create
Add a new test suite which sets
FS_S3A_CONDITIONAL_CREATE_ENABLED=true to the
FS creation configuration, then verify that the request is not
issued with conditional options
1. `opt(conditional create, true) => PathIOException`
2. `opt(conditional create etag, "string") => PathIOException`
3. `opt("fs.s3a.create.performance",true).overwrite(false)` succeeds with no
conditional IO attempted
This test suite should not be skipped if conditional IO is disabled,
because it is the behaviour we require there.
### existing tests
`testIfMatchCreateFileWithoutOverwriteWithPerformanceFlag`
You can't set filesystem configuration options once the FS has been
created...you'll have to work out a different solution.
Look at `ITestCreateFileCost.testCreateFilePerformanceFlag()` to see how to
do it, and how the costs tests have an easy way to make assertions about
statistics.
### everywhere
Add assertions for the open input streams to .haveCapability() of the
conditional create and etag flags, into existing tests, including those where
the performance flag is expected to turn this on.
Why? this allows us to make fast asserts that the relevant flag has been
passed down, without even waiting for a close() to complete and then asserting
on the statistics (these are still important, but a fast failure helps isolate
issues to flag propagation, rather than statistic collection.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
