JunRuiLee commented on PR #99:
URL:
https://github.com/apache/paimon-vector-index/pull/99#issuecomment-5652411183
@jerry-024
On how the API is meant to grow -- the shape is deliberate, so it is worth
writing down what it was measured against.
**What I looked at.** FAISS declares `range_search` on `Index` and takes a
single `radius`. Doris and OpenSearch likewise expose one threshold. Knowhere
takes two (`radius` + `range_filter`). None of them takes SQL operators into
the index layer.
**A two-sided band, not a single radius.** A one-sided API cannot express a
band at all without two calls and a difference, while the second bound costs
nothing when it is `Unbounded` -- which is how the single-threshold case is
written here. So the two-sided form is strictly more expressive at no cost to
the common query, and Knowhere is the precedent that it is not exotic.
**Operators stay out of the index layer.** `DistanceBand` takes cuts, not
`<`/`<=`, so engine-specific vocabulary does not reach the kernels. Callers
who
think in predicates get `DistanceEndpoint` + `CutOperator` as a separate
derivation step rather than as the search signature.
**The result container follows FAISS.** `lims` / `labels` / `distances` is
the
shape of `RangeSearchResult`, because the row count per query is not known in
advance. Unsorted and unpadded, because ordering is the caller's business.
**Two error classes, because the difference is actionable.** `InvalidInput`
means the call is a bug worth fixing; `Unsupported` means this request cannot
be served and the caller should fall back to an exhaustive scan.
**The rule for growing it** is the one your `Auto` comment states, and I have
now applied it three times: nothing ships that no well-formed call can reach.
`RangeSearchWidth`, `limit_reached` and `StopReason`'s two unreachable
variants
are all gone, each to return with the work that produces it. The params and
stats structs keep private fields so that stays additive.
**Two things the later steps have to answer**, and my current answer to both
is
documentation rather than a signature change:
- Metric direction. `admit` is `lower <= value < upper` on the index's own
value. Under L2 the upper bound is the natural cut; under inner product,
where larger means nearer, it is the lower one. Both are expressible, but
the
half-openness is fixed on the upper side and does not follow the metric, so
that needs saying explicitly when cosine and IP land.
- Strictness on the estimator families. `from_endpoints` turns `<= t` into
`< nextafter(t)`, which is exact for IVF-Flat. For a quantized family the
estimate's own error dwarfs that nudge, so the operator is honoured against
the estimate, not against the true distance. The docs should say that
plainly.
Happy to be talked out of any of it -- the bindings step is where these
freeze,
so before then is the cheap time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
