dragonknightk opened a new issue, #11180:
URL: https://github.com/apache/paimon/issues/11180
### Search before asking
- [x] I searched in the issues and found nothing similar. #5681 reports the
same mechanism for the Spark compact procedure, which #6593 already fixed.
### Paimon version
master (9b6c997)
### Compute Engine
Flink
### Minimal reproduce step
```sql
CREATE TABLE T (k INT, v INT);
-- insert 10 rows with 10 separate INSERT statements
ALTER TABLE T SET (
'snapshot.expire.execution-mode' = 'async',
'snapshot.num-retained.min' = '1',
'snapshot.num-retained.max' = '1');
INSERT INTO T VALUES (100, 100); -- in batch mode, or in streaming mode
with a bounded source
SELECT snapshot_id FROM T$snapshots;
```
### What doesn't meet your expectations?
Only the latest snapshot should be retained, but all 11 snapshots remain,
both in batch mode and in streaming mode with a bounded source.
With async mode, `TableCommitImpl` queues maintenance on its executor, and
`close()` calls `shutdownNow()`, which interrupts a running expiration or drops
one that has not started. #6593 made `commit(List)` run maintenance on the
caller thread, which covers Spark. The end-input commit of a Flink bounded job
uses `COMMIT_IDENTIFIER` but goes through `filterAndCommit`, so its maintenance
is still async and is lost when the committer closes right after it. The Flink
remove-unexisting-files action has the same pattern.
### Anything else?
#5681 can likely be closed, since its Spark scenario has been fixed by #6593.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]