github-actions[bot] commented on code in PR #69678:
URL: https://github.com/apache/doris/pull/69678#discussion_r4386524910
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/copier/LogicalPlanDeepCopier.java:
##########
@@ -110,6 +111,8 @@ public Plan
visitLogicalCatalogRelation(LogicalCatalogRelation catalogRelation,
catalogRelation.withRelationId(StatementScopeIdGenerator.newRelationId());
if (context.shouldInvalidatePartitionPruning() && newRelation
instanceof LogicalOlapScan) {
newRelation = ((LogicalOlapScan)
newRelation).withPartitionPruned(false);
+ } else if (context.shouldInvalidatePartitionPruning() && newRelation
instanceof LogicalFileScan) {
+ newRelation = ((LogicalFileScan)
newRelation).withPartitionPruned(false);
Review Comment:
[P2] Preserve effective partition-filter evidence on repeated pruning
This also invalidates unchanged non-PCT file scans in a compensated join. A
reduced input is `Join(Filter(f.dt IN (p1,p2), FactScan), Filter(d.k=1,
FileScan(d)))`, where the fact has p1/p2/p3, p1 is stale, p2 is valid, and
external dimension d has partitions k=1/k=2. The query and MV definition can
both contain d.k=1, so the original dimension scan has selection `{1}` and
`hasPartitionPredicate=true`.
After this reset, `PruneFileScanPartition` re-prunes d.k=1 against that
retained one-entry map. `PartitionPruner.hasEffectivePartitionPredicate`
compares one result with one input and returns false; the new selection
overwrites the previous true flag. `PhysicalPlanTranslator` forwards false, and
when CBO chooses this compensated candidate,
`StmtExecutor.checkBlockRulesByScan` / `SqlBlockRuleMgr.checkLimitations`
rejects the query with "missing partition filter" if require_partition_filter
is enabled, despite the effective filter still being present. Preserve prior
effective-filter evidence, or recompute it against the original pinned
universe, and cover an unchanged filtered scan in compensation.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalFileScan.java:
##########
@@ -378,6 +388,17 @@ public boolean isNativePlanning() {
return nativePlanning;
}
+ /**
+ * Return a copy with the specified partition pruning state.
+ */
+ public SelectedPartitions withPruned(boolean isPruned) {
+ if (nativePlanning || this.isPruned == isPruned) {
+ return this;
+ }
+ return new SelectedPartitions(totalPartitionNum,
selectedPartitions, isPruned,
Review Comment:
[P2] Keep the re-pruning index consistent with the retained selection
Consider `Filter(d IN (Jan1,Feb1)) -> FileScan(selected={Jan1,Feb1},
isPruned=true)` over an Iceberg day-partitioned table that also has Jan2. A
nonaggregate SELECT * MV can be partitioned by date_trunc(d,'month'), with
January invalid and February valid/nonempty. `MaterializedViewOnlyScanRule`
rewrites the bare scan group, leaving the original filter in its parent.
`PartitionCompensator.getNeedCompensatePartitions` compensates the whole
invalid January bucket, so the copied subtree gets a Jan1-or-Jan2 filter.
This copy enables pruning but retains only `{Jan1,Feb1}`. The previous prune
did not retain sorted ranges, so `PruneFileScanPartition` falls back to
`PluginDrivenMvccExternalTable.getSortedPartitionRanges`, which returns the
full pinned index. `PartitionPruner.binarySearchFiltering` returns Jan2 without
intersecting the retained map; `PruneFileScanPartition` then throws "pruned
partition ... is missing in the selected partitions snapshot". The MV-rule
catch discards this otherwise valid scan-only alternative. A filter-plus-scan
alternative may still succeed, so this is a lost optimization rather than
guaranteed query failure. Keep the selection and index coherent when
re-pruning, and add a pruned-subset/monthly-rollup case with binary-search
pruning enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
