csurong opened a new issue, #10967:
URL: https://github.com/apache/paimon/issues/10967

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/paimon/issues) 
and found nothing similar.
   
   
   ### Paimon version
   
   master, commit 24708924fcb169b79620312127486aed4b177ddc (2.2-SNAPSHOT)
   
   ### Compute Engine
   
   Flink 1.20.1, JDK 11. Reproduced with the Flink SQL Procedure and Flink 
Action on local filesystem tables.
   
   ### Minimal reproduce step
   
   Create a primary-key table in the Paimon catalog:
   
   ```sql
   CREATE TABLE word_count (
       word STRING PRIMARY KEY NOT ENFORCED,
       cnt INT
   ) WITH (
       'num-sorted-run.compaction-trigger' = '9999',
       'changelog-producer' = 'input',
       'snapshot.num-retained.min' = '4',
       'snapshot.num-retained.max' = '4',
       'changelog.num-retained.min' = '10',
       'changelog.num-retained.max' = '10'
   );
   ```
   
   Execute each INSERT separately and wait for it to finish, producing ten 
commits:
   
   ```sql
   INSERT INTO word_count VALUES ('0', 0);
   INSERT INTO word_count VALUES ('1', 1);
   INSERT INTO word_count VALUES ('2', 2);
   INSERT INTO word_count VALUES ('3', 3);
   INSERT INTO word_count VALUES ('4', 4);
   INSERT INTO word_count VALUES ('5', 5);
   INSERT INTO word_count VALUES ('6', 6);
   INSERT INTO word_count VALUES ('7', 7);
   INSERT INTO word_count VALUES ('8', 8);
   INSERT INTO word_count VALUES ('9', 9);
   ```
   
   At this point, snapshots 7–10 and separated changelogs 1–6 remain. Update 
the table retention settings without writing any further data:
   
   ```sql
   ALTER TABLE word_count SET (
       'changelog.num-retained.min' = '4',
       'changelog.num-retained.max' = '8'
   );
   
   CALL sys.expire_changelogs(`table` => 'default.word_count');
   ```
   
   The call returns 0 and changelogs 1–6 remain. Now explicitly pass the same 
retention values already configured on the table:
   
   ```sql
   CALL sys.expire_changelogs(
       `table` => 'default.word_count',
       retain_min => 4,
       retain_max => 8
   );
   ```
   
   This call deletes 2 changelogs, leaving 3–6. The latest table still contains 
all ten rows. The same behavior was reproduced through the Flink 
expire_changelogs Action created by ActionFactory.
   
   ### What doesn't meet your expectations?
   
   Calling expire_changelogs with only the table identifier should apply the 
table retention policy. Explicit arguments should override table options, while 
omitted arguments should inherit them.
   
   Currently, the Flink procedure creates a fresh ExpireConfig.Builder and 
fills only the supplied arguments. Missing arguments use builder defaults 
rather than the table options, so eligible changelogs can remain unexpired.
   
   ### Anything else?
   
   The Spark expire_changelogs procedure already uses 
ProcedureUtils.fillInChangelogOptions to inherit table retention options. Flink 
can reuse that helper.
   
   Local before/after verification on real tables: the unmodified Flink 
implementation fails three regression cases covering Action/SQL count retention 
and SQL time retention. With the proposed fix, the complete 
ExpireChangelogsProcedureITCase passes all nine tests.
   
   Compatibility note: calls that omit arguments may start expiring logs 
according to the table policy. Partial overrides can also become invalid if, 
for example, an explicit retain_max is below the inherited retain_min.
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to