FrankChen021 commented on code in PR #20149:
URL: https://github.com/apache/druid/pull/20149#discussion_r3871709691
##########
processing/src/main/java/org/apache/druid/query/filter/FilterSegmentPruner.java:
##########
@@ -164,10 +201,37 @@ public String toString()
'}';
}
+ /**
+ * Adds the filter's {@link RangeSet} for {@code column} to {@code
filterDomain}, resolving through
+ * {@code domainVirtualColumns} to the query's equivalent virtual column if
{@code column} is virtual there.
+ */
+ private void addToFilterDomain(
+ String column,
+ VirtualColumns domainVirtualColumns,
+ Map<String, RangeSet<String>> filterDomain
+ )
+ {
+ final VirtualColumns.Node domainNode =
domainVirtualColumns.getNode(column);
+ if (domainNode != null) {
+ final VirtualColumn queryEquivalent = getQueryEquivalent(domainNode);
+ if (queryEquivalent != null &&
filterFields.contains(queryEquivalent.getOutputName())) {
+ final Optional<RangeSet<String>> optFilterRangeSet =
rangeCache.computeIfAbsent(
+ queryEquivalent.getOutputName(),
+ d -> Optional.ofNullable(filter.getDimensionRangeSet(d))
+ );
+ optFilterRangeSet.ifPresent(rangeSet -> filterDomain.put(column,
rangeSet));
+ }
+ } else if (filterFields.contains(column)) {
Review Comment:
Confirmed the prior same-name physical-column shadowing finding is resolved
by the current guard and regression test. A separate P1 remains at
`processing/src/main/java/org/apache/druid/query/filter/FilterSegmentPruner.java:220`:
query virtual-column dependencies can shadow physical inputs, causing
incorrect cluster-group pruning. Reviewed 2 of 2 changed files.
<!-- mergelens:review -->
##########
processing/src/main/java/org/apache/druid/query/filter/FilterSegmentPruner.java:
##########
@@ -164,10 +201,46 @@ public String toString()
'}';
}
+ /**
+ * Adds the filter's {@link RangeSet} for {@code column} to {@code
filterDomain}, if the filter constrains it.
+ * <p>
+ * If {@code domainVirtualColumns} considers {@code column} virtual, only a
query virtual column with an
+ * equivalent expression can be matched against it, if none exists, nothing
is added and this column is never pruned on.
+ * <p>
+ * Otherwise, {@code column} is a plain physical column, it can only be used
for pruning if it's a non-virtual column in the query.
+ */
+ private void addToFilterDomain(
+ String column,
+ VirtualColumns domainVirtualColumns,
+ Map<String, RangeSet<String>> filterDomain
+ )
+ {
+ final VirtualColumns.Node domainNode =
domainVirtualColumns.getNode(column);
+ if (domainNode != null) {
+ final VirtualColumn queryEquivalent = getQueryEquivalent(domainNode);
Review Comment:
P1 Avoid shadowed query dependencies in VC equivalence
getQueryEquivalent(domainNode) matches a segment-side virtual column without
accounting for query virtual-column dependencies. For example, with segment v0
= dim1 and query VCs dim1 = dim2 plus q = dim1, q can be treated as equivalent
to v0 even though it reads dim2. The pruner can then use a range for q against
the wrong cluster-group value and prune segments that contain matching rows.
Resolve equivalence from the query dependency graph, and add a regression test
for this shadowed-dependency case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
