JingsongLi opened a new pull request, #8849: URL: https://github.com/apache/paimon/pull/8849
## Purpose Complete merge-on-read support for postpone bucket tables across Flink and Spark. ## Changes - Add the Flink batch execution path for postpone merge-on-read, including input assignment, sorting, merging, parallelism inference, and source integration. - Extend the shared core planning and external-sort utilities required by the Flink execution path. - Fix Spark postpone merge-on-read scan state, snapshot pinning, aggregation fallback, and empty-snapshot handling. - Reject unsupported Flink lookup and dynamic-filtering paths explicitly. - Cover the combined `postpone.merge-on-read` and deletion-vector merge-on-read behavior in both Flink and Spark, including actual deletion-vector generation. ## Impact Flink batch reads can opt into a consistent merged view of real-bucket and un-compacted postpone data. Spark reads keep the same semantics without falling back to an ordinary scan or observing a newer snapshot than the selected scan. ## Validation - Flink 1: `PostponeBucketTableITCase#testDeletionVector` — passed with the full static checks enabled. - Flink 2: `PostponeBucketTableITCase#testDeletionVector` — passed. - Spark 3: `PostponeBucketTableTest` — 16 tests passed with the full static checks enabled. - Spark 4: `PostponeBucketTableTest` — 16 tests passed. - `git diff --check` — passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
