JingsongLi commented on PR #968: URL: https://github.com/apache/paimon-rust/pull/968#issuecomment-5935241081
Requirement fit: SUPPORTED. Implementation: FINDINGS. [P1] Preserve live branch references during snapshot expiration (`crates/paimon/src/table/expire_snapshots.rs:222-260`). This PR includes the shared expiration implementation from #965. Its data and manifest protection only considers the current branch's tags and the retained end snapshot. I reproduced: write snapshot 1; create tag t1 and branch b1 from it; delete the main t1 tag; overwrite main with snapshot 2; expire main with retain_min=1. Main still reads row 2, but b1, which read row 1 before expiration, now fails with NotFound for its original manifest list. Branch snapshot/tag metadata remains present and shares physical files with main. Protect all live branch owners before deleting shared data/manifests (or reject expiration when those references cannot be protected). This also violates #964's explicit branch-reference preservation requirement; this is not a claim that Java's implementation behaves differently. [P2] Include actual interrupted snapshot temporary names in orphan cleanup (`crates/paimon/src/table/orphan_files_clean.rs:318`). The writer creates `snapshot/snapshot-<id>.tmp-<uuid>`, but cleanup excludes every name starting `snapshot-`; snapshot listing also ignores this nonnumeric suffix. An old file with the actual writer name survives a real filesystem cleanup with deleted_file_count=0. The test fixture `.snapshot-3.tmp` does not exercise this format. Exclude completed numeric snapshot names specifically and add a test using the writer's temporary format. Validation at 9abaa8deb30986968da880285c8461ae4256136f: all 13 orphan-cleanup tests, 23 expiration tests and 36 DataFusion procedure tests passed. The two additional behavioral probes above fail. Current main (2d0dda18) also has a content conflict in table/mod.rs, so merge integration needs a rebase and rerun. Live cloud and concurrent distributed writers were not exercised locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
