JingsongLi opened a new pull request, #126: URL: https://github.com/apache/paimon-vector-index/pull/126
DiskANN currently cannot serve distance-band queries through the IVF range API. This adds dedicated single-query, batch and Roaring-filtered DiskANN range APIs in Rust, C, C++, Java/JNI and Python. The independent implementation follows Microsoft's [two-stage range-search approach](https://github.com/microsoft/DiskANN/blob/a3e8963c204c631ad09cac155bf6785ad4e5ba8e/diskann/src/graph/search/range_search.rs). It first converges a bounded candidate set, then expands a radius-based frontier. Final membership uses the requested half-open band and persisted F32/F16 vector distances; filters and lower bounds do not block graph navigation. Graph discovery remains approximate, with an explicit exhaustive mode and automatic scanning for unbounded upper cuts and zero cosine queries. Range queries use separate traversal scratch and bounded query-local window caches, batch known I/O windows within adapter limits, and return their own diagnostics. Existing Top-K traversal, caches and statistics are preserved, including the new unified search statistics. Existing IVF parameters, C ABI layouts, C++ result aggregates and Python result dataclass fields remain compatible. The index format is unchanged. Validation on latest main (`7b4b19b`): - Rust workspace: 751 passed, 2 existing ignored; build, Clippy with warnings denied, and formatting checks passed. - Python: 599 passed. C/C++ strict builds with UBSan and Java/JNI tests passed; each language passed the shared 168-case range oracle. - Regression coverage includes uncapped results, metric/layout/PQ/vector-encoding combinations, graph bridges through excluded nodes, numerical errors, working-memory limits, partial I/O failures, bounded remote read batching, legacy result compatibility, and preservation of Top-K results/statistics. - ASF source-header and binary legal-inventory checks passed. Performance evidence: before rebasing onto latest main, an 8-round alternating comparison against the original base (`f6a5558`) reused identical serialized index bytes and a fixed read plan. Recall was identical; median single/batch Top-K QPS changed by +3.7%/+3.1%, with no observed regression. These local measurements do not establish a remote-storage latency guarantee. Added a reproducible range recall/I/O and Top-K interleaving benchmark. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
