J-HowHuang opened a new pull request, #19729: URL: https://github.com/apache/pinot/pull/19729
## Summary `PhysicalColumnIndexContainer` asks an index type for a reader only when the segment stores a buffer for it. An index whose content can be derived from other data of the column, for example from its forward index, therefore has no way to be served: with no stored buffer, the column simply has no reader of that type. This adds an opt-in hook so an index type can still supply one. ## Changes - `IndexReaderFactory#createIndexReaderWithoutStoredIndex(segmentReader, fieldIndexConfigs, metadata, storedReaders)`: called for the index types a column has no stored buffer for. The default returns `null`, so every existing index type behaves exactly as before. - `IndexReaderFactory.StoredIndexReaders`: a view of the column's readers created from stored indexes, passed to the hook so a derived reader can build on them (for example, keep a reference to the forward index reader). - `PhysicalColumnIndexContainer` creates all stored-index readers first, then calls the hook for the missing types, so a derived reader sees every stored reader regardless of numeric index id order. Missing types are tracked in a `long` mask, like the present ones, so the column load allocates nothing extra. ## Testing `PhysicalColumnIndexContainerTest`: - a derived reader is created for a column without a stored index and sees the stored forward reader, but not other derived readers; - a stored index still goes through `createIndexReader` and never through the new hook; - derived readers are closed with the container. The existing container tests and `SegmentPreProcessorTest` pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
