csurong opened a new issue, #10967: URL: https://github.com/apache/paimon/issues/10967
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version master, commit 24708924fcb169b79620312127486aed4b177ddc (2.2-SNAPSHOT) ### Compute Engine Flink 1.20.1, JDK 11. Reproduced with the Flink SQL Procedure and Flink Action on local filesystem tables. ### Minimal reproduce step Create a primary-key table in the Paimon catalog: ```sql CREATE TABLE word_count ( word STRING PRIMARY KEY NOT ENFORCED, cnt INT ) WITH ( 'num-sorted-run.compaction-trigger' = '9999', 'changelog-producer' = 'input', 'snapshot.num-retained.min' = '4', 'snapshot.num-retained.max' = '4', 'changelog.num-retained.min' = '10', 'changelog.num-retained.max' = '10' ); ``` Execute each INSERT separately and wait for it to finish, producing ten commits: ```sql INSERT INTO word_count VALUES ('0', 0); INSERT INTO word_count VALUES ('1', 1); INSERT INTO word_count VALUES ('2', 2); INSERT INTO word_count VALUES ('3', 3); INSERT INTO word_count VALUES ('4', 4); INSERT INTO word_count VALUES ('5', 5); INSERT INTO word_count VALUES ('6', 6); INSERT INTO word_count VALUES ('7', 7); INSERT INTO word_count VALUES ('8', 8); INSERT INTO word_count VALUES ('9', 9); ``` At this point, snapshots 7–10 and separated changelogs 1–6 remain. Update the table retention settings without writing any further data: ```sql ALTER TABLE word_count SET ( 'changelog.num-retained.min' = '4', 'changelog.num-retained.max' = '8' ); CALL sys.expire_changelogs(`table` => 'default.word_count'); ``` The call returns 0 and changelogs 1–6 remain. Now explicitly pass the same retention values already configured on the table: ```sql CALL sys.expire_changelogs( `table` => 'default.word_count', retain_min => 4, retain_max => 8 ); ``` This call deletes 2 changelogs, leaving 3–6. The latest table still contains all ten rows. The same behavior was reproduced through the Flink expire_changelogs Action created by ActionFactory. ### What doesn't meet your expectations? Calling expire_changelogs with only the table identifier should apply the table retention policy. Explicit arguments should override table options, while omitted arguments should inherit them. Currently, the Flink procedure creates a fresh ExpireConfig.Builder and fills only the supplied arguments. Missing arguments use builder defaults rather than the table options, so eligible changelogs can remain unexpired. ### Anything else? The Spark expire_changelogs procedure already uses ProcedureUtils.fillInChangelogOptions to inherit table retention options. Flink can reuse that helper. Local before/after verification on real tables: the unmodified Flink implementation fails three regression cases covering Action/SQL count retention and SQL time retention. With the proposed fix, the complete ExpireChangelogsProcedureITCase passes all nine tests. Compatibility note: calls that omit arguments may start expiring logs according to the table policy. Partial overrides can also become invalid if, for example, an explicit retain_max is below the inherited retain_min. ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
