JingsongLi commented on PR #965: URL: https://github.com/apache/paimon-rust/pull/965#issuecomment-5935442084
Requirement fit: SUPPORTED. Implementation: FINDINGS. [P1] Protect every live branch before deleting shared files (`crates/paimon/src/table/expire_snapshots.rs:222-260`). The protection set is built only from the current branch's tags and retained end snapshot. A real writer/read probe creates snapshot 1, tag t1 and branch b1 from it, deletes the main t1 tag, overwrites main with snapshot 2, then expires main to one snapshot. Main remains readable with row 2, but b1, which read row 1 before expiration, now fails with NotFound for its original manifest list. Branch snapshot/tag metadata remains present, and its files are physically shared with main. Protect data and manifests referenced by every branch owner (not merely add branch tags to the closest-main-tag heuristic), or reject expiration when those references cannot be protected. This violates #964's explicit branch-reference safety rule; no Java-parity claim is needed. #966 further exposes this failure as an automatic commit side effect. [P2] Keep files referenced by persisted long-lived changelogs (`crates/paimon/src/table/expire_snapshots.rs:234-238`, `snapshot_deletion.rs:324-326`). Java's decoupled expiration writes changelog/changelog-<id> immediately before deleting snapshot-<id>, so an interrupted run legitimately leaves both owners. Changelog(Snapshot) retains its changelogManifestList, which Java incremental readers use. A real input-changelog probe persists that valid owner for snapshot 1, commits snapshot 2, and expires to one snapshot: changelog-1 remains but its previously existing changelog manifest list is deleted. Protect those persisted owners or reject this unsupported state before any mutation. Implementing their independent retention policy can remain out of scope; preventing corruption of an existing owner cannot be replaced by documenting that options are ignored. Validation at 695d3890687e477192631bdf59ab24e72ab109df: the original 23 core expiration tests pass; both additional writer/reference probes above fail. The same failures were independently reproduced through the real Python extension in #967. DataFusion procedure integration and read-failure/REST resolution were also checked. Current main conflicts in table/mod.rs, so integration needs a rebase and rerun before merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
