JingsongLi opened a new pull request, #8579: URL: https://github.com/apache/paimon/pull/8579
## Summary Adds snapshot-consistent input planning for bucket-local primary-key vector search. The planner captures one table snapshot and combines its data files, deletion vectors, and matching vector payloads into complete per-bucket search inputs. ## Changes - Add `PrimaryKeyVectorScan` to resolve one snapshot and use it for both data-file planning and index-manifest scanning. - Group all data splits by `(partition, bucket)` while preserving data-file order and deletion-vector alignment, so each search task sees complete bucket state. - Select vector payloads by partition, vector field id, index type, and source metadata to keep different field-scoped indexes isolated. - Add serializable `BucketVectorSearchSplit` transport for a complete bucket `DataSplit` plus its active vector payload metadata. - Validate snapshot consistency, batch-only inputs, duplicate data files, aligned deletion files, and active source-backed payloads during planning. ## Testing - [x] Verify data splits from one bucket are merged with deletion vectors aligned to their source files. - [x] Verify data files and vector payloads are read from the same captured snapshot. - [x] Verify payload selection filters both vector field id and index type. - [x] Verify bucket vector-search splits preserve source metadata through Java serialization. - [x] Run `PrimaryKeyVectorScanTest` on JDK 8: 3 tests passed. - [x] Run Checkstyle, Spotless, Maven enforcers, compilation, and the focused test without `-Pfast-build`. ## Notes - Follows #8576, which materializes primary-key vector results from `IndexedSplit` physical positions. - Executing bucket searches, merging cross-bucket Top-K, and converting candidates to `IndexedSplit` are intentionally left to a follow-up PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
