xiangfu0 commented on code in PR #19539:
URL: https://github.com/apache/pinot/pull/19539#discussion_r4201485779
##########
pinot-broker/src/main/java/org/apache/pinot/broker/routing/segmentpruner/SinglePartitionColumnSegmentPruner.java:
##########
@@ -105,6 +125,35 @@ public Set<String> prune(BrokerRequest brokerRequest,
Set<String> segments) {
return selectedSegments;
}
+ private Set<String> pruneWithPreparedPredicate(Expression filterExpression,
Set<String> segments) {
+ Set<String> selectedSegments = new HashSet<>();
+ List<PreparedPredicate> predicates = new ArrayList<>();
+ for (String segment : segments) {
+ SegmentPartitionInfo partitionInfo = _partitionInfoMap.get(segment);
+ if (partitionInfo == null || partitionInfo ==
SegmentPartitionUtils.INVALID_PARTITION_INFO) {
+ selectedSegments.add(segment);
+ continue;
+ }
+ PartitionFunction function = partitionInfo.getPartitionFunction();
+ int functionHashCode = function.hashCode();
+ PreparedPredicate predicate = null;
+ for (PreparedPredicate candidate : predicates) {
Review Comment:
Implemented in `850bad9f22`: the prepared path now resolves the whole
supported predicate eagerly once per equivalent partition function per prune
call, then uses segment overlap checks. AND intersects, OR unions, unrelated
predicates are unrestricted, and singleton results retain the fast membership
path. Single-valued partition-column validation makes the intersection safe;
preparation errors fall back to the original evaluator rather than discarding
invalid values. This supersedes my earlier replies that kept preparation lazy.
The [fresh benchmark
rerun](https://github.com/apache/pinot/pull/19539#discussion_r4201480263)
compares the previous cached implementation and preparation-disabled control at
256/512/1024/4096/16384 segments, including allocation measurements and paired
rechecks. Nested filters improve about 25–35% at 512+ segments versus before
cached; the 256 repeats show +0.62 µs for IN(16) and +0.73 µs for first-hit IN.
The PR description includes the gains, small-case costs, and inconclusive
controls. No production code or benchmark source is added in this rerun; all 13
Apache CI checks pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
