aokolnychyi commented on code in PR #7389:
URL: https://github.com/apache/iceberg/pull/7389#discussion_r1179623756
##########
api/src/main/java/org/apache/iceberg/actions/RewritePositionDeleteFiles.java:
##########
@@ -28,6 +32,80 @@
public interface RewritePositionDeleteFiles
extends SnapshotUpdate<RewritePositionDeleteFiles,
RewritePositionDeleteFiles.Result> {
+ /**
+ * Enable committing groups of files (see max-file-group-size-bytes) prior
to the entire rewrite
+ * completing. This will produce additional commits but allow for progress
even if some groups
+ * fail to commit. This setting will not change the correctness of the
rewrite operation as file
+ * groups can be compacted independently.
+ *
+ * <p>The default is false, which produces a single commit when the entire
job has completed.
+ */
+ String PARTIAL_PROGRESS_ENABLED = "partial-progress.enabled";
+
+ boolean PARTIAL_PROGRESS_ENABLED_DEFAULT = false;
+
+ /**
+ * The maximum amount of Iceberg commits that this rewrite is allowed to
produce if partial
+ * progress is enabled. This setting has no effect if partial progress is
disabled.
+ */
+ String PARTIAL_PROGRESS_MAX_COMMITS = "partial-progress.max-commits";
+
+ int PARTIAL_PROGRESS_MAX_COMMITS_DEFAULT = 10;
+
+ /**
+ * The entire rewrite operation is broken down into pieces based on
partitioning and within
+ * partitions based on size into groups. These sub-units of the rewrite are
referred to as file
+ * groups. The largest amount of data that should be compacted in a single
group is controlled by
+ * {@link #MAX_FILE_GROUP_SIZE_BYTES}. This helps with breaking down the
rewriting of very large
Review Comment:
optional: Javadoc points to itself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
