xiangfu0 commented on code in PR #19589:
URL: https://github.com/apache/pinot/pull/19589#discussion_r4516368627
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/dociditerators/SVScanDocIdIterator.java:
##########
@@ -142,6 +142,13 @@ public MutableRoaringBitmap applyAnd(BatchIterator
docIdIterator, OptionalInt fi
return result.get();
}
+ @Override
+ public int matchDocIds(int[] docIds, int limit) {
+ _numEntriesScanned += limit;
+ // matchValues already compacts the matching document ids to the front of
the array
+ return _valueMatcher.matchValues(limit, docIds);
Review Comment:
**[should-address] Bound the incoming chunk to the scan matcher's
capacity.** `RestrictedScanDocIdIterator` accepts arbitrary `chunkSize`, but
the dictionary/numeric matcher buffers are allocated from this scan's
constructor batch size, and this call forwards `limit` without bounding it. A
scan with capacity 256 wrapped with a 512-document chunk attempts to read 512
values into a 256-element buffer; the existing fixed-256 benchmark is safe, but
the new API fails for that accepted chunk size. Cap or validate the wrapper
chunk against scan capacity, or split matching into bounded batches, and add
the larger-than-capacity case before wiring streaming into query execution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
