yashmayya opened a new pull request, #18720: URL: https://github.com/apache/pinot/pull/18720
## Summary Re-enables the `JoinPushTransitivePredicates` planner rule by default, reverting the default-disable introduced in #17181. ## Background #17181 disabled this rule by default because `JoinPushTransitivePredicatesRule` could enter an expensive, effectively looping predicate-inference path during planning (`RelMdPredicates#getPredicates` → `RelOptPredicateList.union` → `predicateConstants` → `gatherConstraints`). In certain environments and query workloads this drove broker planning CPU from ~5% to ~70% alongside query timeouts. It was disabled as a default while remaining opt-in via the `usePlannerRules` query option. ## What changed Calcite was upgraded to 1.42.0 in #18658, which includes the fix for [CALCITE-7302](https://issues.apache.org/jira/browse/CALCITE-7302) ("Infinite loop with `JoinPushTransitivePredicatesRule`", apache/calcite#4649). That loop was caused by predicate-inference dedup failing once a conjunction was simplified into a `SEARCH`/`Sarg` form, so the rule kept re-inferring the same predicates. With this fixed, the rule is safe to enable by default again. For reference, the earlier related fix [CALCITE-6432](https://issues.apache.org/jira/browse/CALCITE-6432) / apache/calcite#4346 (another `JoinPushTransitivePredicatesRule` infinite-loop fix) shipped back in Calcite 1.40.0; CALCITE-7302 is the follow-up that 1.42.0 brings in. ## Behavior / blast radius - The rule now fires by default for multi-stage join / semi-join queries that carry a predicate on a join key, pushing the transitively-derived predicate to both sides of the join (e.g. for `a.col1 = b.col1 AND a.col1 = 1`, the rule also adds `b.col1 = 1`). - The Pinot carve-out that does **not** push predicates to the right side under a lookup-join hint is preserved (covered by `QueryCompilationTest#testJoinPushTransitivePredicateLookupJoin`). - The rule remains opt-out: per-query via `skipPlannerRules='JoinPushTransitivePredicates'`, or cluster-wide via the `pinot.broker.mse.planner.disabled.rules` broker config. ## Tests - Updated the affected plan-assertion resources (`ExplainPhysicalPlans.json`, `JoinPlans.json`, `PhysicalOptimizerPlans.json`) to reflect the rule firing, and removed the now-unnecessary `usePlannerRules` opt-in from `QueryCompilationTest#testJoinPushTransitivePredicate`. - Full `pinot-query-planner` suite passes (1263 tests, 0 failures). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
