JingsongLi opened a new pull request, #8569: URL: https://github.com/apache/paimon/pull/8569
## Summary Adds the bucket-local search kernel for primary-key vector indexes. A bucket can now merge ANN results with exact fallback results from active data files that are not covered by ANN yet, while returning physical source positions for the later read path. ## Changes - Introduce a shared `PkVectorSearchResult` containing data file name, zero-based physical row position, and normalized distance, and use it directly from ANN search instead of the unreleased nested candidate type. - Add bounded-memory exact Top-K search for `l2`, `cosine`, and `inner_product`, preserving physical positions across null vectors and applying deletion-vector exclusions. - Add bucket-local search that validates ANN source metadata against active files, avoids rescanning ANN-covered files, forwards deletion vectors and search options, and deterministically merges ANN and exact candidates. ## Testing - [x] Verify multi-source ANN results map to physical data-file positions through the shared result type. - [x] Verify exact search supports all configured metrics and rejects invalid queries. - [x] Verify null vectors and deletion vectors preserve physical row addressing. - [x] Verify bucket search merges ANN and exact fallback Top-K without rescanning covered files. - [x] Run the focused Core suite: 9 tests passed. - [x] Run non-fast Core compilation with Checkstyle, Spotless, and Maven enforcers. ## Notes - Follows #8563, which publishes and restores bucket-level primary-key vector indexes. - Table scan routing, split serialization, physical-row materialization, cross-bucket Top-K, and Flink source integration remain follow-up work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
