JingsongLi opened a new pull request, #124: URL: https://github.com/apache/paimon-vector-index/pull/124
Top-K diagnostics currently differ between DiskANN and IVF-RQ and are not exposed consistently across bindings. Python also lacks the automatic IVF initial filter expansion cap. This change exposes one per-call SearchStats snapshot for all five index families in Rust, C, C++, Java, and Python. - Report executed initial/effective width, candidate work, strategy, fallback reason, adapter read calls/ranges/bytes, and cache hits/misses. Batch counters include every query, progressive round, and DiskANN worker; widths are batch maxima. - Add a bounded, appendable C statistics output prefix and language wrappers while retaining existing C parameter layouts, exported entry points, result types, and Rust algorithm-specific diagnostics. - Forward Python max_initial_filter_expansion_factor through all four extended search paths, retaining existing top-K behavior with older native libraries when optional exports are unavailable. - Normalize unified IVF single-query results to fixed top-K slots before checking progressive completion. Empty initial IVF-PQ rounds now continue expanding, and short filtered results match batch sentinel behavior. The low-level PQ variable-length API is unchanged. - Document counter semantics and add regressions for exact progressive totals, early-finished batch queries, empty/short results, negative IDs, storage request accounting, ABI prefixes, and language bindings. Validation on macOS: - cargo test --workspace - cargo clippy --all-targets --workspace -- -D warnings - cargo fmt --all -- --check; git diff --check; ASF license-header checks - Python: 426 passed, 1 optional range-fixture case skipped because no fixture directory was configured - C and C++ native test executables - Maven tests and Java native behavior tests Additional review probes covered compact/interleaved DiskANN batches exceeding 1,024 queries and forced graph-to-exhaustive fallback. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
