JingsongLi opened a new pull request, #450: URL: https://github.com/apache/paimon-rust/pull/450
## Summary Adds a Rust core batch vector search API so multiple query vectors can be evaluated in one call while preserving one result per input vector. The existing single-vector search builder now delegates to the batch path, keeping the DataFusion UDTF behavior unchanged. ## Changes - Introduce `BatchVectorSearchBuilder` and `Table::new_batch_vector_search_builder()`. - Refactor vector search evaluation to merge and top-k results independently per query vector. - Add batch raw fallback scanning so unindexed row ranges are read once for multiple query vectors. - Add backend batch hooks for Lumina and vindex; Lumina uses its native `n > 1` search path when possible and falls back to per-vector search for filtered or mixed-limit queries. ## Testing - `cargo fmt --all --check` - `git diff --check` - `cargo test -p paimon table::vector_search_builder` - `cargo test -p paimon-datafusion --test read_tables vector_search_tests::test_vector_search_java_vindex_table` - `cargo test -p paimon-datafusion --test read_tables vector_search_tests::test_vector_search_without_matching_index_returns_empty` ## Notes The current DataFusion `vector_search` table function signature and behavior are unchanged. A true Spark-style lateral/table-function join over outer-row vectors should be implemented as a follow-up logical/physical DataFusion extension on top of this core batch API. The full local `vector_search_tests` suite could not complete because the two Lumina query tests require `liblumina_py.dylib`; the vindex and missing-index tests pass, and the Lumina build test is already ignored unless `LUMINA_LIB_PATH` is set. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
