JingsongLi opened a new pull request, #10172: URL: https://github.com/apache/paimon/pull/10172
## Context This draft is an architecture review branch based on #10146 at `c2ee7c1efec31d006b832ea1fca2afe0ff1e8b34`. It includes the original distributed global-index query implementation and the follow-up refactor, so the PR against `master` shows the complete change. ## Refactor - Rename `LazyIndexedSplit` to `IndexQuerySplit` and `GlobalIndexScanPlan` to `GlobalIndexQueryPlan` to describe their responsibilities without implying that split generation is lazy. - Rename the option to `global-index.query-in-reader.enabled` and document that it is a core scan option, including its behavior with Flink dedicated split generation. - Pass split-local row ranges through `GlobalIndexer.createReader` for every index type. Each implementation decides whether to use the pruning hint; callers still clip returned candidates. BTree continues to use the ranges during posting-list reads. - Update serialization references and tests while preserving split type ID `8`. ## Review focus - Ownership and naming of the query plan, split, and option. - The optional range hint at the `GlobalIndexer` boundary. - Recovery semantics when index coverage is incomplete. ## Validation - Core and Flink test sources compiled with the `flink1` profile. - Five focused `GlobalIndexQueryPlanTest` cases passed, covering BTree and Bitmap range forwarding and clipping large results. Mockito required a preloaded Byte Buddy agent in this local environment. - Core Spotless and `git diff --check` passed. ## Open issue before merge The missing-index fallback path may emit rows in a different order from the original split. A restored reader using `recordsToSkip` could then skip valid rows. This needs a correctness fix or an explicit recovery proof before the PR is ready to merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
