xiangfu0 commented on PR #19303:
URL: https://github.com/apache/pinot/pull/19303#issuecomment-5491613686
Added the missing realtime HNSW integration coverage, plus one guard, on
`e63a090a59`.
**`HnswVectorRealtimeTest`** — filtered HNSW search against a *consuming*
segment, where the index is
`MutableVectorIndex`. Nothing covered this before: the other HNSW
integration tests are offline-only, and
`IvfPqVectorRealtimeTest` asserts the IVF_PQ exact-scan fallback.
Two things worth recording, because both were surprises found while making
the test actually exercise the path:
1. **The optional metadata pre-filter is wired far more narrowly than it
looks.** It requires *all* of: the filter
matching at least `EXACT_SCAN_THRESHOLD` (1000) rows, selectivity below
the mid-range cutoff of 0.105, and
every non-vector filter being able to produce a bitmap. My first attempt
used a 1000-row table with a plain
(unindexed) category column and quietly planned as `POST_FILTER_ANN` — it
would have passed as a green test
that proved nothing. The fixture is now sized deliberately (15000 rows,
12 categories -> 1250 matches at
selectivity 0.083) with an inverted index on the filter column, and the
reasoning is in the class Javadoc so
the next person does not have to rediscover it.
2. **`searchMode` is not assertable from `EXPLAIN`.** It is initialized to
`POST_FILTER_ANN` and only reassigned
to `FILTER_THEN_ANN` while the search executes; `EXPLAIN` plans without
executing, so it always reports the
initial value. The test asserts `filterSelectivity` instead, which is
derived from the bitmap the operator
actually received and so genuinely distinguishes a wired pre-filter from
an unwired one.
The tests discriminate — verified, not assumed. Against the unwired plan the
same assertions returned **1 row
instead of 10** and **5 instead of 50**; they pass 6/6 once the bitmap
reaches the operator.
**`HnswVectorIndexReader`** now rejects a null pre-filter bitmap and
short-circuits an empty one. It was the only
one of the five filter-aware readers with neither guard, so a null there
NPE'd deep inside Lucene traversal
rather than failing with the contract message.
Validation on `e63a090a59`: 76 `pinot-segment-local` vector tests,
`HnswVectorRealtimeTest` 6/6, and
spotless/checkstyle/license all clean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
