JingsongLi opened a new pull request, #8576: URL: https://github.com/apache/paimon/pull/8576
## Summary Adds the physical-row materialization path for primary-key vector search results. Bucket-local ANN and exact search hits can now be transported as serializable splits and read from their source data files while preserving the physical row position and vector score. ## Changes - Add `PrimaryKeyVectorDataSplit` for one data file, selected zero-based physical row positions, and their scores, with stable `SplitSerializer` type id 8 and Java serialization support. - Add `PrimaryKeyVectorPositionReader`, which filters raw file batches by selected positions and exposes the source position and score through `ScoreRecordIterator`. - Route vector result splits through a dedicated `KeyValueTableRead` provider into `RawFileSplitRead` when delete rows do not need to be retained. - Intersect ANN-selected positions with file-index and deletion-vector filtering before reading, so only live matching rows are materialized. - Keep scalar predicate pushdown, but disable regular TopN and limit pushdown for vector result splits because applying them before position filtering could incorrectly discard ANN hits. ## Testing - [x] Verify stable split serialization, score preservation, and Java serialization round trips. - [x] Verify physical position and score propagation for both single-batch and multi-batch file readers. - [x] Verify `KeyValueTableRead` routes vector result splits to the raw reader. - [x] Verify a regular read limit does not discard an ANN hit at a higher physical file position. - [x] Run the focused Core suite: 10 tests passed. - [x] Run Checkstyle, Spotless, Maven enforcers, and the focused tests on JDK 8. ## Notes - Follows #8569, which adds bucket-local primary-key vector search and returns data-file plus physical-position results. - Generating these splits from bucket search, cross-bucket Top-K, snapshot scan routing, and Flink source integration remain follow-up work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
