mxm commented on code in PR #15566:
URL: https://github.com/apache/iceberg/pull/15566#discussion_r2924975956
##########
flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java:
##########
@@ -626,6 +633,92 @@ public Builder setSnapshotProperty(String property, String
value) {
return this;
}
+ /**
+ * Enables or disables compaction (rewriting data files) as a post-commit
maintenance task.
+ *
+ * @param enabled whether to enable compaction
+ * @see RewriteDataFilesConfig for the default config.
+ * @deprecated See {@code rewriteDatafiles(..)}
+ */
+ @Deprecated
+ public Builder compaction(boolean enabled) {
+ writeOptions.put(FlinkWriteOptions.COMPACTION_ENABLE.key(),
Boolean.toString(enabled));
+ return this;
+ }
+
+ /**
+ * Enables or disables compaction (rewriting data files) as a post-commit
maintenance task.
+ *
+ * @param enabled whether to enable compaction
+ * @see RewriteDataFilesConfig for the default config.
+ */
+ public Builder rewriteDataFiles(boolean enabled) {
+ writeOptions.put(FlinkWriteOptions.COMPACTION_ENABLE.key(),
Boolean.toString(enabled));
+ return this;
+ }
+
+ /**
+ * Enables or disables compaction (rewriting data files) as a post-commit
maintenance task.
+ *
+ * @param enabled whether to enable compaction
+ * @param config task-specific configuration, see {@link
RewriteDataFilesConfig} for available
+ * keys
+ */
+ public Builder rewriteDataFiles(boolean enabled, Map<String, String>
config) {
+ rewriteDataFiles(enabled);
+ writeOptions.putAll(config);
+ return this;
+ }
+
+ /**
+ * Enables or disables expire snapshots as a post-commit maintenance task.
+ *
+ * @param enabled whether to enable expire snapshots
+ * @see ExpireSnapshotsConfig for the default config.
+ */
+ public Builder expireSnapshots(boolean enabled) {
+ writeOptions.put(FlinkWriteOptions.EXPIRE_SNAPSHOTS_ENABLE.key(),
Boolean.toString(enabled));
Review Comment:
Defaults are hard. These are the current defaults:
ExpireSnapshotsConfig defaults:
- schedule.commit-count: 10
- schedule.data-file-count: 1,000
- schedule.data-file-size: 100 GB
- schedule.interval-second: 3600 (1 hour)
- max-snapshot-age-seconds: none
- retain-last: none
- delete-batch-size: 1,000
- clean-expired-metadata: false
- planning-worker-pool-size: none (shared pool)
DeleteOrphanFilesConfig defaults:
- schedule.commit-count: 10
- schedule.data-file-count: 1,000
- schedule.data-file-size: 100 GB
- schedule.interval-second: 3600 (1 hour)
- min-age-seconds: 259,200 (3 days)
- delete-batch-size: 1,000
- use-prefix-listing: false
- planning-worker-pool-size: none (shared pool)
- equal-schemes: none
- equal-authorities: none
- prefix-mismatch-mode: ERROR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
