Whenever I can create a reproduction for a buggy rule I try to file an issue. But it's not always easy, since we have a custom planner, which is essentially a sequence of Hep planners interspersed with some additional tests to decide whether some rules should be applied.
Moreover, there is no way to combine Quidem tests with custom planners to exhibit incorrect results (I filed an issue about this, but never got the time to work on it). Today the testing of planner rules requires people squinting at EXPLAINED plans, which are very difficult to judge. Having the ability to run end-to-end tests with custom planners would go a long way towards testing properly the rules. Mihai ________________________________ From: Alessandro Solimando Sent: Wednesday, April 2, 2025 12:13 AM To: dev@calcite.apache.org Subject: Re: Newbie question: Choosing rules for HepPlanner? Hi, I share the advise of looking into one of the many OSS databases using Calcite as planner (https://calcite.apache.org/docs/powered_by.html), in my experience it's more an art than science so it requires a lots of experimentation to see what works best in your specific case (e.g., execution engine). Mihai, do we have Jira tickets filed for such unsound rules? It would be very interesting if you could share your experience with such rules for others to benefit and hopefully fix the existing issues. Best regards, Alessandro On Wed, 2 Apr 2025 at 06:57, xiaoheng <xiaoh...@email.cn> wrote: > I think you can start from org.apache.calcite.tools#Program.java, and > check its implementation classes, you will find some example code > constructing Planner and adding rules. And you can also checkout some > open-source project which uses calcite as their optimizer, like Apache > Flink and Apache Hive, you can search for Program implementations in their > codebase and see how they construct planner and adding rules. > > > Hey everyone, > > > > > > I'm new to Apache Calcite and trying to get my head around building a > query optimizer. > > > > > > So far, I've managed to parse a set of SQL queries and convert them into > RelNodes. My understanding is that the next step is often to use a > HepPlanner followed by a VolcanoPlanner. (Is that generally the right > approach?) > > > > > > My main question is about the HepPlanner. It uses rules, and looking > through CoreRules, there are so many! They all seem potentially useful in > different scenarios. How do people typically decide which rules to apply > with HepPlanner? Do I need to manually figure out the best rules based on > the patterns in my specific SQLs, or is there a more standard set or > strategy? > > > > > > Any pointers would be awesome! > > > > > > Thanks, > > > > Liu Yi > > >
