NestDream opened a new issue, #10088:
URL: https://github.com/apache/paimon/issues/10088
### Search before asking
- [x] I searched in the issues and found nothing similar.
### Paimon version
master (54d8596ce). The check is the same in release-1.0 through
release-2.1; `retain_min` has been passed straight through to
`ExpireSnapshotsImpl` since 0.9 (#3335).
### Compute Engine
Flink 1.20.1 (`sys.expire_snapshots` procedure and the `expire_snapshots`
action). The Spark procedure goes through the same
`ProcedureUtils.fillInSnapshotOptions` and `ExpireSnapshotsImpl`.
### Minimal reproduce step
```sql
CREATE TABLE T (id INT, v STRING, PRIMARY KEY (id) NOT ENFORCED)
WITH ('bucket' = '1', 'write-only' = 'true', 'changelog-producer' = 'input');
INSERT INTO T VALUES (1, 'a');
INSERT INTO T VALUES (2, 'b');
INSERT INTO T VALUES (3, 'c');
CALL sys.expire_snapshots(`table` => 'default.T', retain_min => 0,
older_than => '2030-01-01 00:00:00');
SELECT snapshot_id FROM T$snapshots;
SELECT * FROM T /*+ OPTIONS('incremental-between'='2,3',
'incremental-between-scan-mode'='changelog') */;
```
### What doesn't meet your expectations?
The CALL returns `0` and `T$snapshots` still lists 1, 2, 3, but the three
`changelog-*.parquet` files under `bucket-0` are deleted, including the one of
snapshot 3, and the last SELECT fails with `java.io.FileNotFoundException: File
'.../default.db/T/bucket-0/changelog-...-0.parquet' not found`. If snapshot 3
is an `INSERT OVERWRITE` instead, the data files of snapshots 1 and 2 are
deleted and `SELECT * FROM T /*+ OPTIONS('scan.snapshot-id'='2') */` fails the
same way while `T$snapshots` still lists snapshot 2. `retain_min => -1` behaves
the same.
Cause: `ExpireSnapshotsImpl.expire()` only checks `retainMax >= retainMin`.
With `retainMin <= 0` the exclusive end becomes `latestSnapshotId + 1`,
`innerExpireUntil` deletes the changelog files of every snapshot and the data
files marked deleted by later snapshots in the range, then finds that snapshot
`latestSnapshotId + 1` does not exist, returns 0 and never deletes the snapshot
files. The table option `snapshot.num-retained.min` is validated to be at least
1; the procedure argument is not.
Expected: the call is rejected with an argument error before anything is
deleted, the same way `retain_max < retain_min` is.
### Anything else?
Two-line fix: `checkArgument(retainMin >= 1, ...)` next to the existing
check in `ExpireSnapshotsImpl`.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
