xiangfu0 opened a new pull request, #18694:
URL: https://github.com/apache/pinot/pull/18694
## Summary
`JsonMatchFilterOperator` only iterates the matching-docId bitmap
(`BitmapDocIdSet`,
`BitmapCollection`, `getCardinality`) within the segment's lifetime and
never mutates it.
However `JsonIndexReader#getMatchingDocIds` is contractually required to
return a caller-owned
`MutableRoaringBitmap`, so a reader whose posting list is backed by
read-only / memory-mapped
storage must **copy** the posting on every `JSON_MATCH` call — even though
the operator never
writes to it.
This PR adds an additive, backward-compatible read-only variant on the
`JsonIndexReader` SPI:
```java
default ImmutableRoaringBitmap getMatchingDocIdsImmutable(Object filterCtx) {
return getMatchingDocIds(filterCtx);
}
default ImmutableRoaringBitmap getMatchingDocIdsImmutable(String
documentFilter,
String flatDocCountFilter) {
return getMatchingDocIds(documentFilter, flatDocCountFilter);
}
```
and switches `JsonMatchFilterOperator` to call it.
## Contract
The returned bitmap MAY be a read-only view backed by the index's underlying
storage. It is valid
only while the segment/index is held open and **must not be mutated**.
`JsonMatchFilterOperator`
satisfies this — it only iterates the bitmap within the segment's acquired
lifetime.
## Backward compatibility
Purely additive: two `default` methods, no signature changes, no behavior
change for any existing
reader. The default delegates to the owned `getMatchingDocIds`. The stock
`ImmutableJsonIndexReader`
flattens array docIds and remaps each flattened id to a real id, so it keeps
the default (copying)
path.
A reader whose postings are already in real-docId space (no array flattening
/ pre-mapped postings)
can override this to return the posting directly and avoid the per-call
copy. In a downstream
mmap-backed reader this eliminated the copy (reader-level microbenchmark:
the matching-docId fetch
dropped ~20× on a ~360k-match posting), bringing `JSON_MATCH` on a
non-selective JSON path combined
with a selective filter to parity with an equivalent inverted-index column.
## Testing
Existing `JsonMatch`/json-index tests pass. The default path is exercised by
all current readers;
the override path is validated by a downstream reader's query-compatibility
and doc-id-mapping
tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
