Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/24657 )
Change subject: IMPALA-13534: Implement runtime filters on CTEs ...................................................................... Patch Set 13: (2 comments) http://gerrit.cloudera.org:8080/#/c/24657/8/be/src/exec/cte-consumer-node.cc File be/src/exec/cte-consumer-node.cc: http://gerrit.cloudera.org:8080/#/c/24657/8/be/src/exec/cte-consumer-node.cc@249 PS8, Line 249: MaterializeBatch(input_batch, output_batch, &tuple_buf); : } else { : fn(this, input_batch, output_batch, &tuple_buf); : } : } : : i > One other difference vs the scanners is that HdfsScanner implements CheckFi Done http://gerrit.cloudera.org:8080/#/c/24657/8/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java File fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java: http://gerrit.cloudera.org:8080/#/c/24657/8/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java@1457 PS8, Line 1457: : /** : * Assigns pending runtime filters to a CTE consumer node. Filters targeting the CTE : * consumer's output tuple are assigned to 'cteNode'. This is correct because the CTE : * producer materializes data before the hash join build executes, so filters must be : * applied at the consumer level when reading from the LocalExchanger. Assignment to : * scan nodes inside the producer is prevented by the blocking mechanism in : * generateFiltersRecursive(). : */ : private void assignRuntimeFiltersToCTEConsumer(PlannerContext ctx, : CTEConsumerNode cteNode) { : Preconditions.checkState(cteNode.getTupleIds().size() == 1); : TupleId tid = cteNode.getTupleIds().get(0); : if (!runtimeFiltersByTid_.containsKey(tid)) return; : Analyzer analyzer = ctx.getRootAnalyzer(); : boolean disableRowRuntimeFiltering = : ctx.getQueryOptions().isDisable_row_runtime_filtering(); : boolean enable_overlap_filter = enableOverlapFilter(ctx.getQueryOptions()); : TRuntimeFilterMode runtimeFilterMode = ctx.getQueryOptions().getRuntime_filter_mode(); : : for (RuntimeFilter filter : runtimeFiltersByTid_.get(tid)) { : if (filter.isFinalized()) continue; : if (cteProducerBlockedFilters_.containsKey(filter.getFilterId())) continue; : if (filter.getType() == TRuntimeFilterType.MIN_MAX && !enable_overlap_filter) { : continue; : } : // CTE consumer filters are never partition-bound, so skip in row-filtering mode. : if (disableRowRuntimeFiltering) continue; : Expr targetExpr = computeTargetExpr(filter, tid, analyzer); : if (targetExpr == null) continue; : boolean isLocalTarget = isLocalTargetForPlanNode(filter, cteNode); : > Based on the profiles I'm looking at, we seem to have extra min_max filters I sorted min_max filters before bloom filters, which is something we otherwise do for parquet scanners. I took it into account somewhat, but didn't fully implement enable_overlap_filter support yet. -- To view, visit http://gerrit.cloudera.org:8080/24657 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic877fb590187826f828da6a27bf274465c381e8e Gerrit-Change-Number: 24657 Gerrit-PatchSet: 13 Gerrit-Owner: Michael Smith <[email protected]> Gerrit-Reviewer: Aleksandr Efimov <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Comment-Date: Tue, 18 Aug 2026 00:11:13 +0000 Gerrit-HasComments: Yes
