michaelbraun opened a new pull request, #5272: URL: https://github.com/apache/calcite/pull/5272
## Summary `HepPlanner` applies a rule collection by trying every rule against every vertex, so rule application attempts greatly outnumber matches. Avoidable work sits on that path — some of it on every attempt, some only on attempts that cannot match. Paid on **every** attempt, whether or not it matches: * `applyRule` tested `graph.vertexSet().contains(vertex)` on each call. Both call sites try every rule in the collection against the same vertex object, in a loop that only exits early on an actual match, so graph membership cannot change within that loop. A vertex visited with 100 rules did 100 set lookups where one suffices. The test moves to the two callers, which also return early when the rule collection is empty, so the hoisted test is not paid when there is no rule to apply. Paid on attempts whose operand **cannot match**, the large majority: * `applyRule` allocated the `bindings` list and the `nodeChildren` map before calling `matchOperands`, which rejects on its first statement (`operand.matches(rel)`), so both were allocated and discarded. * The `ConverterRule` and `CommonRelSubExprRule` branches ran before the operand was tested, traversing a vertex's parents for attempts that could never match. `applyRule` now tests the operand before doing anything else. `matchOperands` is split into a thin entry point that makes that test and `matchOperandsWithOperandMatched` which assumes it, so the test is not repeated for the attempts that pass it. The matching logic itself is unchanged, and recursive descent still goes through the checking entry point. ## Measurements `HepQueryVarietyBenchmark`, added here, plans queries converted from SQL through `CoreRules` that fire, timing `setRoot` plus `findBestExp`. Two independent dimensions: `ruleCount` is how many attempts each vertex draws, `firingPercent` is the share of the rule set able to rewrite these queries. Percent faster than unmodified `HepPlanner`, mean over four query shapes: | firing% | 26 rules | 100 rules | |---|---|---| | 0 | 25.3% | 44.2% | | 50 | 18.1% | 28.8% | | 100 | 16.6% | 23.9% | Per query at the least favourable setting, 100% firing with 26 rules: `FILTER_PROJECT` 26.6%, `AGGREGATE` 20.6%, `MIXED_DEEP` 11.9%, `JOIN5` 7.5%. Every cell of the grid improves, from 7.5% to 48.5%. Gains rise with rule count, because more attempts become cheaper, and fall as more of the rule set can fire, because transformation work grows and dilutes the saving. They remain substantial at 100% firing because the membership test is saved whether or not a rule matches. The non-firing portion of the rule set is drawn from real rules for `INTERSECT`, `MINUS`, `MATCH_RECOGNIZE`, `TABLESAMPLE` and exchange, which none of these queries contain — real rules rather than synthetic ones, so the added attempts reject at the operand depths real rules reject at, several having operands with children and so rejecting below the top level. Plans are identical at every rule count, confirming the non-firing portion never fires. Figures are means over three rounds. Each candidate was bracketed between two runs of the baseline jar within the same round and compared against their mean, because the machine used shows run-to-run drift that JMH's within-run error bars understate; the same-jar position control stayed under 2% for all but two cells, against effects several times larger. `ubenchmark` has `LargePlanBenchmark`, which drives `HepPlanner` over large synthetic trees, but no benchmark that plans realistic queries through rules that actually fire, which is what sizing this change needed. ## Test plan - [x] `HepPlannerTest`, `RelOptRulesTest`, `MaterializedViewRelOptRulesTest`, `CombineRelOptRulesTest` — 1064 tests, 0 failures - [x] `checkstyleMain`, `checkstyleJmh`, `autostyleCheck` clean - [ ] full `./gradlew build` 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
