Hello, The reasons for the planner rules configuration can be found here: CALCITE-3923 <https://issues.apache.org/jira/browse/CALCITE-3923>. See also the email thread [DISCUSS] Refactor how planner rules are parameterized <https://lists.apache.org/thread.html/rfdf6f9b7821988bdd92b0377e3d293443a6376f4773c4c658c891cf9%40%3Cdev.calcite.apache.org%3E> .
Cordialement / Best Regards, *Thomas Rebele, PhD* | R&D Developer | Germany | www.tibco.com On Tue, Apr 12, 2022 at 6:10 PM Gavin Ray <[email protected]> wrote: > I don't have any weight behind my opinion or experience, > but anything that lowers the barrier to entry to Calcite for newcomers is a > huge win in my mind. > > I assume the reason for the changes was because codegen improved > performance? > > Could it make sense to allow both options, the easy/less-performant way for > people who want to experiment and learn the ropes, > and the codegen path for productionizing the final rules you come up with? > > Or does this make matters worse, trying to support two API's > > On Tue, Apr 12, 2022 at 6:25 AM Vladimir Ozerov <[email protected]> > wrote: > > > Hi folks, > > > > Rules are an essential part of the Calcite-based query optimizers. A > > typical optimizer may require dozens of custom rules that are created by > > extending some Apache Calcite interfaces. > > > > During the last two years, there were two major revisions of how rules > are > > created: > > > > 1. In early 1.2x versions, the typical approach was to use > > RelOptRuleOperand with a set of helper methods in a builder-like > > pattern. > > 2. Then, we switched to the runtime code generation. > > 3. Finally, we switched to the compile-time code generation with the > > Immutables framework. > > > > Every such change requires the downstream projects to rewrite all their > > rules. Not only does this require time to understand the new approach, > but > > it may also compromise the correctness of the downstream optimizer > because > > the regression tracking in query optimizers is not trivial. > > > > I had the privilege to try all three approaches, and I cannot get rid of > > the feeling that every new approach is more complicated than the previous > > one. I understand that this is a highly subjective statement, but when I > > just started using Apache Calcite and knew very little about it, I was > able > > to write rule patterns by simply looking at the IDE JavaDoc pop-ups and > > code completion. When the RuleConfig was introduced, every new rule > always > > required me to look at some other rule as an example, yet it was doable. > > Now we also need to configure the project build system to write a single > > custom rule. > > > > At the same time, a significant fraction of the rules are pretty simple. > > E.g., "operator A on top of operator B". If some additional configuration > > is required, it could be added via plain rules fields, because at the end > > of the day the rule instance is not more than a plain Java object. > > > > A good example is the FilterProjectTransposeRule. What now takes tens of > > lines of code in the Config subclass [1] (that you hardly could write > > without a reference example), and ~500 LOC in the generated code that you > > get through additional plugin configuration [2] in your build system, > could > > have been expressed in a dozen lines of code [3] in Apache Calcite > 1.22.0. > > > > My question is - are we sure we are going in the right direction in terms > > of complexity and the entry bar for the newcomers? Wouldn't it be better > to > > follow the 80/20 rule, when simple rules could be easily created > > programmatically with no external dependencies, while more advanced > > facilities like Immutables are used only for the complex rules? > > > > Regards, > > Vladimir. > > > > [1] > > > > > https://github.com/apache/calcite/blob/calcite-1.30.0/core/src/main/java/org/apache/calcite/rel/rules/FilterProjectTransposeRule.java#L208-L260 > > [2] > > > > > https://github.com/apache/calcite/blob/calcite-1.30.0/core/build.gradle.kts#L215-L224 > > [3] > > > > > https://github.com/apache/calcite/blob/calcite-1.22.0/core/src/main/java/org/apache/calcite/rel/rules/FilterProjectTransposeRule.java#L99-L110 > > >
