liming30 commented on code in PR #1823:
URL: https://github.com/apache/incubator-paimon/pull/1823#discussion_r1306895915
##########
paimon-common/src/main/java/org/apache/paimon/CoreOptions.java:
##########
@@ -194,6 +194,26 @@ public class CoreOptions implements Serializable {
.defaultValue(Duration.ofHours(1))
.withDescription("The maximum time of completed snapshots
to retain.");
+ public static final ConfigOption<ExpireExecutionMode>
SNAPSHOT_EXPIRE_EXECUTION_MODE =
+ key("snapshot.expire.execution-mode")
Review Comment:
For a table with a particularly large number of files, simply increasing the
number of threads may not be able to delete expired files in time. In our
production scenario, a table is divided into partitions by day, and 10,000+
files are generated each commit, and each partition has about 5 million files.
When the partition expires, it is difficult to delete all within the checkpoint
interval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
