heye1005 commented on PR #7715: URL: https://github.com/apache/paimon/pull/7715#issuecomment-5100509512
> The current `early-abort` behavior, which lacks a manifest, only protects `main`: `LocalOrphanFilesClean` constructs `liveSnapshots` only for `DEFAULT_MAIN_BRANCH`, and Flink/Spark also use the same `liveOnMainBranch` gating. > > * However, the orphan cleaner’s `validBranches()` scans all active branches; since branches have independent snapshot directories, writes and snapshot expiration can also be performed via the branch table. When new snapshots are generated concurrently on branches and old snapshots are deleted due to expiration, stale-list/missing-manifest race conditions may also occur. > * Because a branch’s `missingManifest` is `null`, a read failure is treated as an empty result, which may still cause data files referenced by active branches to be marked as orphans. Recommendation: **All active branch snapshots should adopt the same `missing-manifest` abort behavior as the main branch; tags can remain on a best-effort basis**, and additional regression tests for concurrent branch commits and expirations should be implemented. Thanks for the review — fixed by building liveSnapshots for every active branch across local/Flink/Spark so all branch snapshots share the same missing-manifest abort behavior, kept tags best-effort also, all tests and spot passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
