github-actions[bot] commented on code in PR #69292:
URL: https://github.com/apache/doris/pull/69292#discussion_r4334000233
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/ExpressionTranslator.java:
##########
@@ -280,9 +282,22 @@ public Expr visitMatch(Match match, PlanTranslatorContext
context) {
MatchPredicate matchPredicate = new MatchPredicate(op,
match.left().accept(this, context),
match.right().accept(this, context),
match.getDataType().toCatalogDataType(),
NullableMode.DEPEND_ON_ARGUMENT, invertedIndex,
match.nullable(), analyzer);
+ matchPredicate.setCollectScore(collectsScore(match.getScoreId(),
context));
return matchPredicate;
}
+ // A MATCH or SEARCH of the SELECT that calls score() adds to the score of
the scan that it filters,
+ // see CheckScoreUsage, so that scan has to evaluate it. Any other one
only filters rows.
+ private static boolean collectsScore(Optional<ExprId> predicateScoreId,
PlanTranslatorContext context) {
+ if (!predicateScoreId.isPresent()) {
+ return !context.getScanScoreId().isPresent();
+ }
+ if (!predicateScoreId.equals(context.getScanScoreId())) {
+ throw new
AnalysisException(BindScoreToOlapScan.UNEVALUATED_PREDICATE_ERROR);
Review Comment:
[P2] Make inferred copies of scoring predicates filter-only
A supported same-type text equijoin can acquire a tagged predicate on a
second scan:
```sql
SELECT a.id, score() AS s FROM t a JOIN t2 b ON a.c1=b.c1
WHERE a.c1 MATCH 'apple' OR a.c1 MATCH 'red';
```
Binding correctly adds the score column only to scan a. `InferPredicates`
then passes the single-slot OR and join equality to `InferPredicateByReplace`;
`PredicatesCollector.visitOr` records it, and replacement infers the same OR on
b.c1. `Match.withChildren` preserves a's score id on that copy. The resulting
reduced tree has `Join(Filter(tagged OR)->ScoredScan a, Filter(inferred tagged
OR)->Scan b)`. Translating b hits this exception because b has no scan score
id, so a valid query is rejected. Clear scoring ownership on inferred
filter-only copies, or prevent this inference for scoring expressions, while
retaining the original predicates' scoring contribution.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/CheckScoreUsage.java:
##########
@@ -18,98 +18,187 @@
package org.apache.doris.nereids.rules.rewrite;
import org.apache.doris.nereids.exceptions.AnalysisException;
-import org.apache.doris.nereids.rules.Rule;
-import org.apache.doris.nereids.rules.RuleType;
-import org.apache.doris.nereids.trees.expressions.Alias;
+import org.apache.doris.nereids.jobs.JobContext;
+import org.apache.doris.nereids.properties.OrderKey;
+import org.apache.doris.nereids.trees.expressions.ExprId;
import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.Match;
+import org.apache.doris.nereids.trees.expressions.NamedExpression;
+import org.apache.doris.nereids.trees.expressions.StatementScopeIdGenerator;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Score;
+import org.apache.doris.nereids.trees.expressions.functions.scalar.Search;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.doris.nereids.trees.plans.logical.LogicalAggregate;
-import org.apache.doris.nereids.trees.plans.logical.LogicalOlapScan;
+import org.apache.doris.nereids.trees.plans.logical.LogicalApply;
+import org.apache.doris.nereids.trees.plans.logical.LogicalCTEProducer;
+import org.apache.doris.nereids.trees.plans.logical.LogicalFilter;
import org.apache.doris.nereids.trees.plans.logical.LogicalProject;
+import
org.apache.doris.nereids.trees.plans.logical.LogicalRecursiveUnionAnchor;
+import
org.apache.doris.nereids.trees.plans.logical.LogicalRecursiveUnionProducer;
+import org.apache.doris.nereids.trees.plans.logical.LogicalSetOperation;
+import org.apache.doris.nereids.trees.plans.logical.LogicalSort;
+import org.apache.doris.nereids.trees.plans.logical.LogicalSubQueryAlias;
+import org.apache.doris.nereids.trees.plans.logical.LogicalView;
+import org.apache.doris.nereids.trees.plans.visitor.CustomRewriter;
import com.google.common.collect.ImmutableList;
-import org.apache.logging.log4j.LogManager;
-import org.apache.logging.log4j.Logger;
+import com.google.common.collect.ImmutableSet;
import java.util.List;
+import java.util.Optional;
/**
- * Check score function usage in project and aggregate without proper
optimization context.
+ * Tie every score() call to the MATCH and SEARCH predicates it scores.
*
- * This rule ensures that score() function is used only in contexts where it
can be optimized,
- * requiring WHERE clause with MATCH function, ORDER BY and LIMIT.
+ * <p>score() is the relevance of a row for the MATCH and SEARCH predicates in
the WHERE clause of
+ * its own SELECT. This rewriter runs first in rewrite, while subqueries,
views and CTEs still have
+ * their own plan nodes. It gives every SELECT that calls score() a new score
id and marks with it
+ * the score() calls of the SELECT and the MATCH and SEARCH predicates of its
filters.
+ * BindScoreToOlapScan later binds the calls to the scan that those predicates
filter.
+ *
+ * <p>A MATCH or SEARCH without a score id only filters rows: it adds nothing
to the score of its
+ * scan, even when a filter of an outer SELECT is pushed down onto that scan.
*/
-public class CheckScoreUsage implements RewriteRuleFactory {
- private static final Logger LOG =
LogManager.getLogger(CheckScoreUsage.class);
+public class CheckScoreUsage implements CustomRewriter {
+ public static final String NO_SCORING_PREDICATE_ERROR =
+ "score() requires a MATCH or SEARCH predicate in the WHERE clause
of the same SELECT";
+ public static final String UNSUPPORTED_CLAUSE_ERROR = "score() can only be
used in the SELECT list "
+ + "and in the WHERE, GROUP BY, HAVING, QUALIFY and ORDER BY
clauses";
@Override
- public List<Rule> buildRules() {
- return ImmutableList.of(
- logicalProject(any())
- .when(project -> {
- boolean hasScore = hasScoreFunction(project);
- boolean isOptimized = isScoreAlreadyOptimized(project);
- LOG.debug("Project check - hasScore: " + hasScore + ",
isOptimized: " + isOptimized);
- return hasScore && !isOptimized;
- })
- .then(project -> {
- throw new AnalysisException(
- "score() function requires WHERE clause with MATCH
function, "
- + "ORDER BY and LIMIT for optimization");
- }).toRule(RuleType.CHECK_SCORE_USAGE),
-
- logicalAggregate(any())
- .when(this::hasScoreInAggregate)
- .then(agg -> {
- throw new AnalysisException(
- "score() function cannot be used in aggregate
functions. "
- + "score() requires WHERE clause with MATCH
function, "
- + "ORDER BY and LIMIT for optimization");
- }).toRule(RuleType.CHECK_SCORE_USAGE)
- );
+ public Plan rewriteRoot(Plan plan, JobContext jobContext) {
+ if (!containsScore(plan)) {
+ return plan;
+ }
+ return tagSelect(plan);
+ }
+
+ /** Whether any plan node of the tree calls score() in its own
expressions. */
+ public static boolean containsScore(Plan plan) {
+ return plan.anyMatch(node -> ((Plan) node).getExpressions().stream()
+ .anyMatch(expression -> expression.containsType(Score.class)));
}
- private boolean hasScoreFunction(LogicalProject<?> project) {
- return project.getProjects().stream()
- .anyMatch(projection -> {
- if (projection instanceof Alias) {
- return containsScoreFunction(((Alias)
projection).child());
- }
- return false;
- });
+ // A SELECT consists of the plan nodes from its root down to the roots of
its subqueries, views,
+ // CTE producers and set operation inputs, which are SELECTs of their own.
+ private static boolean startsSelect(Plan parent, int childIndex) {
+ return parent instanceof LogicalSubQueryAlias || parent instanceof
LogicalView
+ || parent instanceof LogicalCTEProducer || parent instanceof
LogicalSetOperation
+ || parent instanceof LogicalRecursiveUnionAnchor || parent
instanceof LogicalRecursiveUnionProducer
+ || parent instanceof LogicalApply && childIndex == 1;
}
- private boolean hasScoreInAggregate(LogicalAggregate<?> agg) {
- boolean hasScoreInOutput = agg.getOutputExpressions().stream()
- .anyMatch(this::containsScoreFunction);
+ private Plan tagSelect(Plan root) {
+ SelectSummary select = new SelectSummary();
+ summarize(root, select);
+ if (!select.callsScore) {
+ return tag(root, Optional.empty());
+ }
+ if (!select.hasScoringPredicate) {
+ throw new AnalysisException(NO_SCORING_PREDICATE_ERROR);
+ }
+ if (select.unsupportedClause) {
+ throw new AnalysisException(UNSUPPORTED_CLAUSE_ERROR);
+ }
+ return tag(root, Optional.of(StatementScopeIdGenerator.newExprId()));
+ }
- boolean hasScoreInGroupBy = agg.getGroupByExpressions().stream()
- .anyMatch(this::containsScoreFunction);
+ private void summarize(Plan plan, SelectSummary select) {
+ if (plan.getExpressions().stream().anyMatch(expression ->
expression.containsType(Score.class))) {
+ select.callsScore = true;
+ select.unsupportedClause |= !canCallScore(plan);
+ }
+ if (plan instanceof LogicalFilter && ((LogicalFilter<?>)
plan).getConjuncts().stream()
+ .anyMatch(conjunct -> conjunct.containsType(Match.class,
Search.class))) {
+ select.hasScoringPredicate = true;
+ }
+ for (int i = 0; i < plan.arity(); i++) {
+ if (!startsSelect(plan, i)) {
+ summarize(plan.child(i), select);
+ }
+ }
+ }
- return hasScoreInOutput || hasScoreInGroupBy;
+ // HAVING and QUALIFY are filters by now. Join conditions are rejected by
CheckAnalysis.
+ private static boolean canCallScore(Plan plan) {
+ return plan instanceof LogicalProject || plan instanceof LogicalFilter
+ || plan instanceof LogicalAggregate || plan instanceof
LogicalSort;
}
- private boolean containsScoreFunction(Expression expr) {
- if (expr instanceof Score) {
- return true;
+ private Plan tag(Plan plan, Optional<ExprId> scoreId) {
+ ImmutableList.Builder<Plan> newChildren =
ImmutableList.builderWithExpectedSize(plan.arity());
+ boolean changed = false;
+ for (int i = 0; i < plan.arity(); i++) {
+ Plan child = plan.child(i);
+ Plan newChild = startsSelect(plan, i) ? tagSelect(child) :
tag(child, scoreId);
+ changed |= newChild != child;
+ newChildren.add(newChild);
}
- return expr.children().stream().anyMatch(this::containsScoreFunction);
+ Plan newPlan = changed ? plan.withChildren(newChildren.build()) : plan;
+ return scoreId.isPresent() ? tagExpressions(newPlan, scoreId.get()) :
newPlan;
}
- private boolean isScoreAlreadyOptimized(LogicalProject<?> project) {
- Plan child = project.child();
- if (child instanceof LogicalOlapScan) {
- LogicalOlapScan scan = (LogicalOlapScan) child;
- return scan.getVirtualColumns().stream()
- .anyMatch(virtualCol -> {
- if (virtualCol instanceof Alias) {
- Expression childExpr = ((Alias)
virtualCol).child();
- return containsScoreFunction(childExpr);
- }
- return false;
- });
+ private Plan tagExpressions(Plan plan, ExprId scoreId) {
+ if (plan.getExpressions().stream()
+ .noneMatch(expression -> expression.containsType(Score.class,
Match.class, Search.class))) {
+ return plan;
}
- return false;
+ if (plan instanceof LogicalFilter) {
+ LogicalFilter<?> filter = (LogicalFilter<?>) plan;
+ return filter.withConjuncts(filter.getConjuncts().stream()
+ .map(conjunct -> tagScoreAndPredicates(conjunct, scoreId))
Review Comment:
[P1] Keep HAVING predicates out of the WHERE score
`Analyzer` has already run `HavingToFilter` and `QualifyToFilter`, so
tagging every LogicalFilter also marks later-clause MATCH/SEARCH as score
contributors. For example:
```sql
SELECT c1, max(score()) AS s FROM t
WHERE c1 MATCH 'apple'
GROUP BY c1 HAVING c1 MATCH 'red';
```
This reaches tagging as `Filter(MATCH red) -> Aggregate(c1,max(score())) ->
Filter(MATCH apple) -> Scan`. Both MATCHs get the same score id. Binding
resolves the aggregate's score from its apple subtree; later
`PushDownFilterThroughAggregation` moves the grouping-key red predicate down to
the scan, where translation enables collection for both. The returned score
includes red BM25 even though only the WHERE predicate defines it (with the
usage fixture, 1.4508327 instead of 0.4700036). Preserve clause origin when
tagging and leave HAVING/QUALIFY MATCH/SEARCH filter-only; add a grouped HAVING
regression that checks the score value.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java:
##########
@@ -1543,7 +1544,19 @@ public PlanFragment visitPhysicalFilter(PhysicalFilter<?
extends Plan> filter, P
// i.e, the physical filter's nereids's id will be mapped
to final plan's scan node
addPlanRoot(inputFragment, planNode, filter);
}
- addConjunctsToPlanNode(filter, planNode, context);
+ // The scan fills its score column only after it evaluated its
own conjuncts, so a
+ // conjunct on the score is evaluated above the scan.
+ Set<Expression> scoreConjuncts = planNode instanceof
OlapScanNode
+ && filter.child(0) instanceof PhysicalOlapScan
+ ? conjunctsOnScore(filter.getConjuncts(),
(PhysicalOlapScan) filter.child(0))
+ : ImmutableSet.of();
+ addConjunctsToPlanNode(filter,
Sets.difference(filter.getConjuncts(), scoreConjuncts),
+ planNode, context);
+ if (!scoreConjuncts.isEmpty()) {
+ planNode = createSelectNode(filter, planNode, context);
+ addPlanRoot(inputFragment, planNode, filter);
+ addConjunctsToPlanNode(filter, scoreConjuncts, planNode,
context);
Review Comment:
[P1] Disable ANN truncation before applying residual score filters
On the new `test_score_filter_ann` fixture, consider:
```sql
SELECT id, l2_distance_approximate(embedding, [1.0,1.0]) AS d
FROM test_score_filter_ann
WHERE c1 MATCH 'apple' AND score() < 0.5
ORDER BY d LIMIT 1;
```
The reduced plan is `TopN(d,1) -> Project -> Filter(MATCH,__score<0.5) ->
Scan`. `PushDownVectorTopNIntoOlapScan` still sets ANN limit 1, while this
block moves the score predicate into a Select above the scan.
`_prepare_score_column_materialization` has no pushed range for `<`, and
`_apply_ann_topn_predicate` sees no remaining scanner predicate, so ANN keeps
nearest id 2. The Select drops it for its score above 0.5, returning no rows
although farther id 1 qualifies. The two candidates out of three also exceed
the default small-candidate fallback percentage. Disable ANN truncation when a
residual predicate reads the score, or evaluate that predicate before ANN
selection, and cover this upper-bound case.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/BindScoreToOlapScan.java:
##########
@@ -0,0 +1,292 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.rewrite;
+
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.jobs.JobContext;
+import org.apache.doris.nereids.properties.OrderKey;
+import org.apache.doris.nereids.trees.expressions.Alias;
+import org.apache.doris.nereids.trees.expressions.ExprId;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.Match;
+import org.apache.doris.nereids.trees.expressions.NamedExpression;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.expressions.functions.scalar.Score;
+import org.apache.doris.nereids.trees.expressions.functions.scalar.Search;
+import org.apache.doris.nereids.trees.plans.Plan;
+import org.apache.doris.nereids.trees.plans.RelationId;
+import org.apache.doris.nereids.trees.plans.logical.LogicalAggregate;
+import org.apache.doris.nereids.trees.plans.logical.LogicalFilter;
+import org.apache.doris.nereids.trees.plans.logical.LogicalOlapScan;
+import org.apache.doris.nereids.trees.plans.logical.LogicalProject;
+import org.apache.doris.nereids.trees.plans.logical.LogicalSort;
+import org.apache.doris.nereids.trees.plans.visitor.CustomRewriter;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.function.Function;
+
+/**
+ * Bind every score() call to the score column of the scan whose rows its
MATCH and SEARCH
+ * predicates filter.
+ *
+ * <p>CheckScoreUsage marked each score() call and the MATCH and SEARCH
predicates of the same
+ * SELECT with one score id. This rewriter runs after views and CTEs are
inlined, and before window
+ * expressions are extracted and filters are pushed down. For each score id it
finds the one olap
+ * scan whose columns those predicates read, adds to the scan a hidden score
column computed for
+ * every row the scan returns, and replaces the score() calls with that
column, passing it through
+ * the projections in between. From then on the score is an ordinary column
for windows, joins and
+ * filter push down. PushDownScoreRangeIntoOlapScan and
PushDownScoreTopNIntoOlapScan only reduce
+ * how much of it the scan has to compute.
+ */
+public class BindScoreToOlapScan implements CustomRewriter {
+ public static final String UNRESOLVED_SCAN_ERROR = "score() requires the
MATCH and SEARCH predicates "
+ + "of its WHERE clause to filter the columns of one table, not of
a set operation, an aggregation "
+ + "or a CTE that is used more than once";
+ public static final String SHARED_SCAN_ERROR = "score() is called by two
SELECTs whose MATCH or SEARCH "
+ + "predicates filter the same table, which computes only one
score: call score() in one of them";
+ public static final String SCORE_NOT_AVAILABLE_ERROR = "score() is not
available above GROUP BY: "
+ + "use it in an aggregate function or as a grouping key";
+ // Thrown by ExpressionTranslator, which finds where each predicate ends
up.
+ public static final String UNEVALUATED_PREDICATE_ERROR = "score() requires
the scan of its table to "
+ + "evaluate each MATCH and SEARCH predicate of its WHERE clause,
which it cannot for a predicate in an "
+ + "OR with conditions on other tables, or on the result of a
LIMIT, a window function or the nullable "
+ + "side of an outer join";
+
+ private final Map<RelationId, ExprId> scanScoreIds = new HashMap<>();
+ private final Map<RelationId, Alias> scoreColumns = new HashMap<>();
+
+ @Override
+ public Plan rewriteRoot(Plan plan, JobContext jobContext) {
+ if (!CheckScoreUsage.containsScore(plan)) {
+ return plan;
+ }
+ return bind(plan, ImmutableMap.of());
+ }
+
+ // scope maps every score id that an ancestor calls to the scan that
computes it.
+ private Plan bind(Plan plan, Map<ExprId, RelationId> scope) {
+ Map<ExprId, RelationId> childScope = scope;
+ for (ExprId scoreId : scoreIds(plan)) {
+ if (!childScope.containsKey(scoreId)) {
+ RelationId scan = resolveScan(plan, scoreId);
+ ExprId boundScoreId = scanScoreIds.putIfAbsent(scan, scoreId);
+ if (boundScoreId != null && !boundScoreId.equals(scoreId)) {
+ throw new AnalysisException(SHARED_SCAN_ERROR);
+ }
+ Map<ExprId, RelationId> newScope = new HashMap<>(childScope);
+ newScope.put(scoreId, scan);
+ childScope = newScope;
+ }
+ }
+ ImmutableList.Builder<Plan> newChildren =
ImmutableList.builderWithExpectedSize(plan.arity());
+ boolean changed = false;
+ for (Plan child : plan.children()) {
+ Plan newChild = bind(child, childScope);
+ changed |= newChild != child;
+ newChildren.add(newChild);
+ }
+ Plan newPlan = changed ? plan.withChildren(newChildren.build()) : plan;
+ if (newPlan instanceof LogicalOlapScan) {
+ return addScoreColumn((LogicalOlapScan) newPlan);
+ }
+ if (newPlan instanceof LogicalProject) {
+ newPlan = passScoreColumns((LogicalProject<?>) newPlan, scope);
+ }
+ return replaceScore(newPlan, childScope);
+ }
+
+ private static Set<ExprId> scoreIds(Plan plan) {
+ Set<ExprId> scoreIds = new LinkedHashSet<>();
+ for (Expression expression : plan.getExpressions()) {
+ for (Score score :
expression.<Score>collect(Score.class::isInstance)) {
+ scoreIds.add(score.getScoreId().get());
+ }
+ }
+ return scoreIds;
+ }
+
+ // The olap scan whose rows the MATCH and SEARCH predicates with the score
id filter.
+ private static RelationId resolveScan(Plan plan, ExprId scoreId) {
+ Set<RelationId> scans = new HashSet<>();
+ collectScoredScans(plan, scoreId, scans);
+ if (scans.size() != 1) {
+ throw new AnalysisException(UNRESOLVED_SCAN_ERROR);
+ }
+ return scans.iterator().next();
+ }
+
+ private static void collectScoredScans(Plan plan, ExprId scoreId,
Set<RelationId> scans) {
+ if (plan instanceof LogicalFilter) {
+ Plan input = plan.child(0);
+ for (Expression conjunct : ((LogicalFilter<?>)
plan).getConjuncts()) {
+ for (Expression predicate : conjunct.<Expression>collect(e ->
definesScore((Expression) e, scoreId))) {
+ if (predicate instanceof Search) {
+ // search() reads its fields from the scan below it,
see RewriteSearchToSlots.
+ collectOlapScans(input, scans);
+ } else {
+ for (Slot slot : predicate.getInputSlots()) {
+ scans.add(findScan(input, slot.getExprId())
+ .orElseThrow(() -> new
AnalysisException(UNRESOLVED_SCAN_ERROR)));
+ }
+ }
+ }
+ }
+ }
+ for (Plan child : plan.children()) {
+ collectScoredScans(child, scoreId, scans);
+ }
+ }
+
+ private static boolean definesScore(Expression expression, ExprId scoreId)
{
+ Optional<ExprId> predicateScoreId = Optional.empty();
+ if (expression instanceof Match) {
+ predicateScoreId = ((Match) expression).getScoreId();
+ } else if (expression instanceof Search) {
+ predicateScoreId = ((Search) expression).getScoreId();
+ }
+ return predicateScoreId.isPresent() &&
predicateScoreId.get().equals(scoreId);
+ }
+
+ private static void collectOlapScans(Plan plan, Set<RelationId> scans) {
+ if (plan instanceof LogicalOlapScan) {
+ scans.add(((LogicalOlapScan) plan).getRelationId());
+ }
+ for (Plan child : plan.children()) {
+ collectOlapScans(child, scans);
+ }
+ }
+
+ // The olap scan that outputs the slot as one of its columns, following
projections that pass
+ // or rename it. Empty when another operator computes the slot, e.g. an
aggregation.
+ private static Optional<RelationId> findScan(Plan plan, ExprId slotId) {
+ if (plan instanceof LogicalOlapScan) {
+ return plan.getOutputExprIdSet().contains(slotId)
+ ? Optional.of(((LogicalOlapScan) plan).getRelationId()) :
Optional.empty();
+ }
+ if (plan instanceof LogicalProject) {
+ for (NamedExpression project : ((LogicalProject<?>)
plan).getProjects()) {
+ if (!project.getExprId().equals(slotId)) {
+ continue;
+ }
+ if (project instanceof Slot) {
+ return findScan(plan.child(0), slotId);
+ }
+ Expression producer = ((Alias) project).child();
+ return producer instanceof Slot
+ ? findScan(plan.child(0), ((Slot)
producer).getExprId()) : Optional.empty();
+ }
+ return Optional.empty();
+ }
+ if (plan instanceof LogicalAggregate) {
+ return Optional.empty();
+ }
+ for (Plan child : plan.children()) {
+ if (child.getOutputExprIdSet().contains(slotId)) {
+ return findScan(child, slotId);
+ }
+ }
+ return Optional.empty();
+ }
+
+ private LogicalOlapScan addScoreColumn(LogicalOlapScan scan) {
+ ExprId scoreId = scanScoreIds.get(scan.getRelationId());
+ if (scoreId == null) {
+ return scan;
+ }
+ Alias scoreColumn = new Alias(new Score().withScoreId(scoreId),
"__score");
+ scoreColumns.put(scan.getRelationId(), scoreColumn);
+ // A zero score limit scores every row the scan returns.
+ return scan.appendVirtualColumnsAndTopN(ImmutableList.of(scoreColumn),
Review Comment:
[P1] Materialize the bound score before runtime-filter evaluation
Binding score as an ordinary scan output also makes it eligible for join
runtime filters. A triggering reduced plan is:
```text
HashJoin(__score = k; BLOOM RF from k)
Filter(id % 2 = 0) -> ScoredScan(MATCH, __score=Score())
BuildScan(k containing matching positive scores)
```
When the RF arrives before `ScanLocalState::open` normalizes conjuncts, it
is pushed as a common expression. `SegmentIterator::_mark_common_expr_states`
expands the score VirtualSlotRef to the leaf scalar Score, so it never marks
the score ordinal for the common-expression read phase. The physical `id % 2`
predicate enables that phase, which runs the RF before deferred output columns
are read. The score column is still ColumnNothing, so
`VirtualSlotRef::execute_column_impl` evaluates the underlying placeholder
Score(), returning zero rather than the BM25 value prepared in its column
iterator. A positive-score Bloom filter therefore discards rows that should
join. Keep score-dependent runtime filters above score materialization, or
ensure their score column is read before common-expression filtering; cover a
score-key join with an additional physical residual predicate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
