juntaozhang opened a new issue, #8720: URL: https://github.com/apache/paimon/issues/8720
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation Chain table streaming read works in two phases: 1. Phase 1 (starting / full load): reads the latest snapshot partition per group plus later delta partitions. 2. Phase 2 (incremental): continuously reads new commits from the delta branch. By default, Phase 1 is lightweight: snapshot and delta partitions are read as separate splits, and anchor-based chain merging is skipped. This means cross-branch deletes and updates are not reflected in the starting snapshot. Example: 1. Snapshot branch has key=100 at dt=20250901. 2. Delta branch receives a delete for key=100 at dt=20250902. 3. A streaming job starts with the default lightweight Phase 1. Result: downstream still receives key=100 from the snapshot split, because the delete in the delta split is not merged with the snapshot at startup. The delta delete therefore never removes the snapshot record, and the stale snapshot row remains visible forever. `compact_chain_table` can merge delta deletes into the snapshot branch and solve this, but there is an operational gap between the delta delete and the next compaction. During that gap, the starting snapshot remains incorrect. ### Solution >ALTER TABLE t SET ('chain-table.streaming.merge-snapshot' = 'true'); - Default: false — keeps the existing lightweight Phase 1 behavior unchanged. - When enabled: Phase 1 performs anchor-based chain merging. For each group, it merges the latest snapshot partition with delta partitions whose chain key is strictly greater than the snapshot chain key. This produces a correct starting snapshot that already reflects cross-branch deletes and updates. - The trade-off is a heavier startup scan, which can be mitigated by periodically running `compact_chain_table`. ### Anything else? _No response_ ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
