RyanLin5967 opened a new issue, #10883: URL: https://github.com/apache/paimon/issues/10883
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version 2.1.0 and master ### Compute Engine Spark 4.1.4 (spark-sql) on Java 21, with paimon-spark-4.1_2.13 and the `paimon` catalog on a local warehouse as in the Spark quick start. The Java API does the same through `deleteTag` and `newExpireSnapshots` on the branch table. ### Minimal reproduce step Currently, a branch created from a tag starts on a copy of main's snapshot, and `expire_snapshots` on the branch checks only the branch's own tags, so once the branch's copy of the tag is gone it deletes files main still reads. Keeping that copy leaves main readable. ```sql USE paimon; CREATE TABLE t (k INT); INSERT INTO t VALUES (1), (2); CALL sys.create_tag(table => 't', tag => 't1'); CALL sys.create_branch(table => 't', branch => 'b', tag => 't1'); INSERT INTO `t$branch_b` VALUES (3); CALL sys.delete_tag(table => '`t$branch_b`', tag => 't1'); CALL sys.expire_snapshots(table => '`t$branch_b`', retain_max => 1, retain_min => 1); CREATE TABLE u (k INT); INSERT INTO u VALUES (1), (2); CALL sys.create_tag(table => 'u', tag => 't1'); CALL sys.create_branch(table => 'u', branch => 'b', tag => 't1'); INSERT INTO `u$branch_b` VALUES (3); CALL sys.expire_snapshots(table => '`u$branch_b`', retain_max => 1, retain_min => 1); -- quit and start a new spark-sql session, since this one still reads t from its cache USE paimon; SELECT * FROM t; -- java.io.FileNotFoundException: File '.../t/manifest/manifest-list-...-0' not found, should be 1 and 2 SELECT * FROM t VERSION AS OF 't1'; -- the same FileNotFoundException, though main's tag t1 is still there SELECT * FROM u; -- 1 and 2, so with b's copy of t1 kept the same expiration leaves main alone ``` ### What doesn't meet your expectations? Both reads of `t` return 1 and 2, since main's snapshot 1 never expired and main's tag `t1` still exists. ### Anything else? `replace_tag` on `b`, or `b`'s copy expiring through `time_retained`, does the same. On a primary-key table, a `compact` on `b` first also deletes a data file main's newer snapshots read. ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
