shyjsarah opened a new pull request, #72: URL: https://github.com/apache/paimon-vector-index/pull/72
## Purpose Automatic IVF batch search currently doubles `nprobe` for the entire batch whenever any query returns fewer than the required number of results. A small number of difficult queries can therefore force already-complete queries to be searched repeatedly at every expansion round. ## Changes - Track incomplete queries independently during automatic IVF batch expansion. - Pack and retry only incomplete queries in later rounds. - Scatter retried results back to their original positions in the batch output. - Apply the behavior to filtered and unfiltered IVF Flat, IVF SQ, IVF PQ, and IVF RQ batch search paths. - Keep explicit/fixed `nprobe` searches as one unchanged full-batch call. - Preserve the no-copy fast path when the first automatic-search round completes the batch. This change intentionally does **not** reuse previously scanned IVF lists. Retried queries still search `0..nprobe`; incremental new-list-only scanning will be handled separately. ## Tests - `cargo fmt --all -- --check` - `cargo test --workspace` - `cargo clippy --workspace --all-targets -- -D warnings` - `mvn -f java/pom.xml test` - JNI release build and native validation, panic-boundary, and handle-safety tests on macOS -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
