clintropolis commented on code in PR #19903:
URL: https://github.com/apache/druid/pull/19903#discussion_r3731179106
##########
server/src/main/java/org/apache/druid/server/coordinator/rules/PartialLoadRule.java:
##########
@@ -99,12 +104,49 @@ public void run(DataSegment segment, SegmentActionHandler
handler)
PartialLoadProfile.forRequest(result.wrappedLoadSpec(),
result.fingerprint()),
getTieredReplicants()
);
- } else {
- // Matcher does not apply, but the rule still applies because
onCannotMatch == FULL_LOAD (FALL_THROUGH would
- // have caused appliesTo to return false, so run wouldn't be invoked).
Route through the regular full-load
- // handler.
- handler.replicateSegment(segment, getTieredReplicants());
+ return;
}
+ // Matcher does not apply, but the rule still does — FALL_THROUGH would
have made appliesTo return false, so run
+ // wouldn't have been invoked. How much of the segment to make resident is
onCannotMatch's call.
+ switch (onCannotMatch) {
+ case LOAD_ON_DEMAND -> handler.replicateSegment(segment,
getTieredReplicants());
+ case BASE_LOAD -> replicateWholly(
+ segment,
+ handler,
+ PartialBaseTableLoadSpec.wireForm(segment.getLoadSpec(),
PartialBaseTableLoadSpec.FINGERPRINT),
+ PartialBaseTableLoadSpec.FINGERPRINT
+ );
+ case FULL_LOAD -> replicateWholly(
Review Comment:
yea, this is a bit of a rough edge but I don't think i'm going to change it
in this PR and since this functionality is still undocumented and a bit
experimental i think its ok for now. Historicals currently lack the machinery
to do sticky eagerly loads of non-rangeable segments since they currently are
stored in different cache entries
(`PartialSegmentMetadataCacheEntry`/`PartialSegmentBundleCacheEntry` vs
`CompleteSegmentCacheEntry` for non-rangeables), so this would take some work
to do to add that weak hold machinery to prevent eviction and honor the rule,
or, they would need to be stored as 'static' entries in the cache and managed
that way, both of which are not directly related to the changes in this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
