clintropolis opened a new pull request, #18710: URL: https://github.com/apache/druid/pull/18710
After #18683, in virtual storage mode there are no longer 'missing' segments after the initial timeline operation to obtain the `DataSegment` objects for the given set of `SegmentDescriptor`, so we can simplify things in such as way as to both give callers additional flexibility on how they handle loading on demand (not really utilized in this PR). To model this, a new `LeafSegementsBundle` has been added that holds the result of partitioning a list of `DataSegmentAndDescriptors` into: * segments which are already loaded in the cache (for which it obtains a `SegmentReference` to place a hold on them) * segments which can be loaded on demand (via virtual storage mode) * segments which are not in the cache and not loadable on demand (e.g. there was no `DataSegment` in the timeline in virtual storage mode, or the segment has been dropped before `getCachedSegment` could be called in regular mode). Two new methods has been added to `SegmentManager`,`getSegmentsBundle` to build a `LeafSegmentsBundle`, and `canLoadSegmentsOnDemand` to indicate that a `SegmentCacheManager` can support loading a segment on demand if it was not present in the cache already. `ServerManager` has been transitioned to use these methods, even though it still downloads all 'load on demand' segments up front before building query runners. This change also lets us streamline the handling of 'missing' segments in `ServerManager`. (e.g. those which are dropped from a historical after the broker has already picked them and issued a query and are added to the header as a 'missing' segment for the broker to retry). Instead of handling missing segments 'inline' and using a `ReportTimelineMissingSegmentQueryRunner`, because we are using `LeafSegmentsBundle` we can just split the handling of these segments out as soon as we make the bundle and directly add them to the missing segments of the response context. `ReportTimelineMissingSegmentQueryRunner` has been removed, since the only remaining uses -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
