steveloughran commented on code in PR #3289:
URL: https://github.com/apache/hadoop/pull/3289#discussion_r891151772
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:
##########
@@ -3970,63 +4062,64 @@ InitiateMultipartUploadResult initiateMultipartUpload(
/**
* Perform post-write actions.
- * <p></p>
+ * <p>
* This operation MUST be called after any PUT/multipart PUT completes
* successfully.
- * <p></p>
- * The actions include:
- * <ol>
- * <li>
- * Calling
- * {@link #deleteUnnecessaryFakeDirectories(Path)}
- * if directory markers are not being retained.
- * </li>
- * <li>
- * Updating any metadata store with details on the newly created
- * object.
- * </li>
- * </ol>
+ * <p>
+ * The actions include calling
+ * {@link #deleteUnnecessaryFakeDirectories(Path)}
+ * if directory markers are not being retained.
* @param key key written to
* @param length total length of file written
* @param eTag eTag of the written object
* @param versionId S3 object versionId of the written object
+ * @param putOptions put object options
*/
@InterfaceAudience.Private
- @Retries.RetryTranslated("Except if failOnMetadataWriteError=false, in which"
- + " case RetryExceptionsSwallowed")
- void finishedWrite(String key, long length, String eTag, String versionId) {
+ @Retries.RetryExceptionsSwallowed
+ void finishedWrite(
+ String key,
+ long length,
+ String eTag,
+ String versionId,
+ PutObjectOptions putOptions) {
LOG.debug("Finished write to {}, len {}. etag {}, version {}",
key, length, eTag, versionId);
- Path p = keyToQualifiedPath(key);
Preconditions.checkArgument(length >= 0, "content length is negative");
- // kick off an async delete
- CompletableFuture<?> deletion;
- if (!keepDirectoryMarkers(p)) {
- deletion = submit(
- unboundedThreadPool, getActiveAuditSpan(),
- () -> {
- deleteUnnecessaryFakeDirectories(
- p.getParent()
- );
- return null;
- });
- } else {
- deletion = null;
+ if (!putOptions.isKeepMarkers()) {
Review Comment:
async was to mix the delete call with the s3guard update for minor
performance benefits. without s3guard, no possibilities of overlapping IO, just
risk of waiting for a spare thread. it has become counterproductive
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
