bgaborg commented on a change in pull request #951: HADOOP-15183. S3Guard store
becomes inconsistent after partial failure of rename
URL: https://github.com/apache/hadoop/pull/951#discussion_r292969539
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/CommitOperations.java
##########
@@ -520,6 +533,120 @@ public void jobCompleted(boolean success) {
statistics.jobCompleted(success);
}
+ /**
+ * Begin the final commit.
+ * @param path path for all work.
+ * @return the commit context to pass in.
+ * @throws IOException failure.
+ */
+ public CommitContext initiateCommitOperation(Path path) throws IOException {
+ return new CommitContext(writeOperations.initiateCommitOperation(path));
+ }
+
+ /**
+ * Commit context.
+ *
+ * It is used to manage the final commit sequence where files become
+ * visible. It contains a {@link BulkOperationState} field, which, if
+ * there is a metastore, will be requested from the store so that it
+ * can track multiple creation operations within the same overall operation.
+ * This will be null if there is no metastore, or the store chooses not
+ * to provide one.
+ *
+ * This can only be created through {@link #initiateCommitOperation(Path)}.
+ *
+ * Once the commit operation has completed, it must be closed.
+ * It must not be reused.
+ */
+ public final class CommitContext implements Closeable {
Review comment:
This could go to a separate file, so CommitOperations could be shorter and
easier to overview.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
