joooger opened a new pull request, #5344: URL: https://github.com/apache/calcite/pull/5344
## Jira Link [CALCITE-4542](https://issues.apache.org/jira/browse/CALCITE-4542) ## Changes Proposed With the top-down optimizer, `PhysicalNode#derive` was never called for a physical rel whose traits satisfy none of the `OPTIMIZING` subsets. A typical case is a rel created with distribution `ANY` while its parent requires `SINGLETON`, when the rel is implemented after its own subset was already optimized. For example, a second implementation rule for the same logical rel can fire from `TopDownRuleDriver#convert` while another subset of the set is being optimized. `onProduce` then schedules the rel only for the required subset. There, `getOptimizeInputTask` replaces the rel with its pass-through copy, and that copy is never derived because it is in `passThroughCache`. If `passThrough` returns `null`, the rel is skipped entirely. As a result, alternatives derived from the inputs' delivered traits were never generated: the planner chose the plan from the JIRA, or failed with `CannotPlanException` when the rel cannot pass the requirement through. `getOptimizeInputTask` now also optimizes the inputs of the original rel and derives its traits: * in the rel's own subset, so that the inputs are not pruned by the upper bound of a subset the rel does not belong to; * whether or not pass-through succeeds; * at most once per rel. `clearProcessed` resets this after set merges, because the rel's inputs may then deliver new traits. Pass-through handling itself is unchanged. Removing `rel = passThroughRel` (#2487) is not enough. The pass-through rel is registered with `applying == null`, so its inputs would no longer be optimized: on `TopDownOptTest`, 39 of the 195 pass-through rels keep inputs that are never optimized. With top-down enabled for the whole `core` suite, `JdbcAdapterTest#testScalarSubQuery` and `#testSubQueryWithSingleValue` then fail on an assertion in `EnumerableMergeJoin`. It also does not cover the case where `passThrough` returns `null`. Tests: `TopDownDeriveTest`. * `testDeriveRelThatPassesThrough` covers the case from the JIRA. * `testDeriveRelThatDoesNotPassThrough` covers `passThroughTraits` returning `null`. * Both fail without the fix. `testPassThroughStillChosen` checks that the pass-through plan still wins when it is cheaper. The whole `core` suite run with `calcite.planner.topdown.opt=true` has the same set of failures with and without this change. Those failures are reference plans that differ under top-down. The planner runs about 10% more tasks on `TopDownOptTest` (14431 vs. 13148). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
