JingsongLi opened a new pull request, #8701: URL: https://github.com/apache/paimon/pull/8701
## Summary Align Python primary-key global-index queries with Java's source-backed scalar-index path. Primary-key BTREE and bitmap results are now interpreted as physical positions within their source data files instead of global row IDs. ## Changes - Add Java-compatible primary-key index definitions and source metadata, including modified UTF-8 serialization and index-manifest persistence. - Plan active sorted-index groups by partition, bucket, data level, and source file, with conservative fallback for incomplete or invalid payloads. - Evaluate shared index queries once per source group and partition results into file-local positions. - Convert evaluated positions into snapshot-scoped `IndexedSplit` instances while preserving deletion-file alignment. - Route primary-key batch scans through the source-backed index path while retaining the existing global-row-ID path for append-only and data-evolution tables. - Match Java fallback behavior for missing coverage, reader failures, invalid positions, excessive ranges, non-raw splits, and reader-close failures. ## Testing - [x] Primary-key index definition and Java-compatible source-metadata tests. - [x] Source-group planning, shared-query localization, invalid-position fallback, and close-failure tests. - [x] Indexed-split conversion and deletion-file alignment tests. - [x] Existing global-index build, update, vector-filter, manifest, and primary-key reader regressions. - [x] `131 passed, 12 subtests passed` in the focused Python suite. - [x] `pyflakes`, Python compile checks, and `git diff --check`. ## Notes The Java implementation is the behavioral reference. Append-only and data-evolution global-index query behavior is unchanged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
