yunfengzhou-hub opened a new pull request, #8423: URL: https://github.com/apache/paimon/pull/8423
### Purpose This PR adds support for Flink lookup join on Chain Table. When used as a dimension table, the lookup join reflects the latest state of the chain table: the most recent snapshot partition per group, combined with delta partitions that come after it. #### Bug fixes in existing paths - **`ChainTableStreamScan.withBucketFilter()`**: Bucket filter was not propagated to inner scans nor stored for replay on local scans created in `planStarting()`. This caused bucket filter to be silently dropped during streaming read. - **`ChainGroupReadTable.copy()` methods**: Branch options were not properly prepared — `scan.mode` from the parent table could propagate to branch tables (conflicting with explicit snapshot pinning), and bucket values could be lost during `copy(TableSchema)`. Added `prepareBranchOptions()` to consistently handle branch name, bucket restoration, and chain-table-level directive stripping across all copy methods. - **`ChainTableUtils`**: Added `validateChainTableForIncrementalRead()` to centralize validation that the delta branch uses `DEDUPLICATE` merge engine for incremental read paths (streaming read and lookup join). #### New lookup join capabilities - **`LookupFileStoreTable`**: Added chain table awareness. When the wrapped table is a chain table, `newStreamScan()` returns `ChainTableStreamScan`. Added validation to reject partition keys in join condition and unsupported cache modes. - **`ChainTableStreamScan`**: Added `pinnedOptions()` helper to set both `scan.snapshot-id` and `scan.mode=from-snapshot` when pinning branch tables, preventing conflicts with `scan.mode=latest-full` inherited from the lookup join path. - **`ChainSplit`**: Added `toString()` for debugging. - **Documentation**: Added Lookup Join section to chain table docs. ### Tests Added comprehensive test coverage in `FlinkChainTableITCase`, including basic lookup join, bucket shuffle, delta-only/snapshot-only scenarios, constraint validation, incremental refresh, checkpoint/restore, predicate pushdown, and merge engine rejection. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
