jerry-024 commented on issue #97: URL: https://github.com/apache/paimon-vector-index/issues/97#issuecomment-5628259875
The feature makes sense: a native range API removes fixed-K truncation and provides a natural way to express distance-band queries. I think this can proceed with two clarifications in the issue: 1. **Top-K plus post-filtering is not an equivalent fallback.** The roadmap suggests this when a family does not support range search, but it reintroduces the problem described in the motivation. Even with exact search, if 100 vectors satisfy the band and K is 10, at most 10 can be returned. An unsupported family should return a distinguishable error; callers may explicitly opt into a capped fallback. Queries requiring complete results need an execution path that preserves that requirement, such as an exhaustive scan. 2. **Exact distance computation and complete result coverage are separate properties.** IVF-Flat computes exact distances for the vectors it visits, but when `nprobe < nlist`, matching vectors in unprobed lists can still be missed. Removing K eliminates truncation; it does not by itself make the search exhaustive. The issue already explains quantization-related incompleteness, but should also state this limitation for IVF-Flat. The documentation in #99 already makes this distinction, so aligning the issue with it should be sufficient. A small reference note: this project's IVF-RQ uses RaBitQ, whereas the table names FAISS `IndexIVFResidualQuantizer`. Since the table is explicitly a reference for semantics rather than implementation, this is not a blocker, but clarifying the distinction would avoid confusing RaBitQ with residual quantization. These are clarifications to the contract, not objections to approximate range search or the phased implementation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
