> We probably need a new Quidem command to fire the rules and generate the > before and after plans.
https://issues.apache.org/jira/browse/CALCITE-6335 Quidem tests should allow specifying optimization passes to apply to programs This issue is implemented. Mihai ________________________________ From: Julian Hyde <[email protected]> Sent: Friday, February 20, 2026 11:00 AM To: [email protected] <[email protected]> Subject: Re: [DISCUSS] Stop testing rules inside RelOptRulesTest I have been pondering changes to RelOptRulesTest, but with some different goals in mind. Sure, large files are a bit unruly. But so are lots of small files: * It’s easy to forget to add a file to a change. * If contributors don’t choose good names for the small files, which they often don’t, it makes it difficult to find what you need. * When you move from a large file to many small files you are replacing search in an editor with grep. Search is a bit more powerful, because you can see multiple lines as you search. Regarding conflicts, I have two concrete suggestions: * Move away from XML. The multi-line “</inner></outer><outer name=“foo”><inner>” pattern that happens between blocks confuses diff. * Add lint checks to ensure that elements of a file are strictly ordered. In my Morel project, I use LintTest to ensure that test methods are sorted alphabetically. See https://github.com/hydromatic/morel/issues/316. I would be happy to bring this over to Calcite. Now, the bigger goals. Java’s role in data ecosystem is declining, with Python and Rust increasing year-over-year and C/C++ holding steady. People developing on those platforms don’t consider Calcite because it is Java. Calcite provides less a runtime than a set of rules, and those rules could be expressed in any programming language. (If you’ve been following my work on Morel, you will know that I am implementing the same language specification in both Java and Rust. The Java implementation of Morel uses Calcite but the Rust implementation currently cannot.) AI makes it fairly straightforward to translate code from one language to another, provided that there is a comprehensive test suite. Certain components (for example implementation of built-in functions and dialect-specific SQL generation) could even be vibe-coded, meaning we don’t look at the generated code, only whether the tests pass. What are the requirements for that test suite? First, the same tests need to be runnable from any language. Second, the tests need to be easy for humans to review, find, and add comments to, because they are our core intellectual property. (In Morel, idempotent, language-independent test scripts have been very effective for generating implementations and porting them between programming languages.) Which components of Calcite would be useful in other languages? Runtime for SQL built-in functions, validation for SQL built-in functions, dialect-specific SQL generation, and planner rules. So, here is my proposal for RelOptRulesTest: Convert RelOptRulesTest.java and RelOptRulesTest.xml to a handful of Quidem (.iq) files. A typical file would be between 100 and 1,000 lines. Each file would deal with a particular area (say join rules) and would have a series of test cases, each with a good, useful description. A test case would be a region of the file that contains a description, a SQL query, a collection of rules, and plans before and after. We probably need a new Quidem command to fire the rules and generate the before and after plans. (Implicit is a bigger proposal, namely to port Calcite to Rust and Python. But we can talk about that in another thread.) Julian > On Feb 20, 2026, at 4:05 AM, Stamatis Zampetakis <[email protected]> wrote: > > Just a small precision because few people mentioned losing history. > Even if the tests are moved the history is not gonna be lost it will > just be a bit more difficult to follow. > > Best, > Stamatis > > On Fri, Feb 20, 2026 at 9:42 AM jensen <[email protected]> wrote: >> >> I agree to freeze >> "RelOptRulesTest". My suggestion is to avoid migrating the tests here either >> step by step or all at once, as that could result in losing important Git >> information. This is because we might need to know who originally added a >> particular test case. >> >> >> >> Best regards, >> >> Zhen Chen >> >> ---- Replied Message ---- >> | From | Ruben Q L<[email protected]> | >> | Date | 02/20/2026 16:26 | >> | To | [email protected] | >> | Cc | | >> | Subject | Re: [DISCUSS] Stop testing rules inside RelOptRulesTest | >> I think this is a good idea. >> >> I agree with Stamatis proposal of "freezing" RelOptRuleTest and from now on >> adding new rule tests in separate, dedicated files. >> Perhaps when adding a new test for rule X (in its dedicated file), the PR >> contributor could examine RelOptRuleTest, see if there are existing tests >> in there for said rule X, and "migrate" them into the new file (so that we >> make RelOptRuleTest lighter little by little). >> >> Best, >> Ruben >> >> >> >> On Fri, Feb 20, 2026 at 8:10 AM Stamatis Zampetakis <[email protected]> >> wrote: >> >>> I don't intend to split the existing RelOptRuleTest class as this would be >>> quite a bit of work with a few shortcomings as well. >>> >>> The proposal here is just for new tests. The majority of the tests are >>> aiming a single rule/transformation so going forward we can have a separate >>> class for the rule under test and put new cases there. >>> >>> I am currently working on a new rule so the respective PR may serve as an >>> example to demonstrate concretely the idea. I will share it here once I >>> finish it. >>> >>> Best, >>> Stamatis >>> >>> On Fri, Feb 20, 2026, 8:37 AM Cancai Cai <[email protected]> wrote: >>> >>>> I agree with splitting the tests, but the question is how we should split >>>> them, which is perhaps the most important point to discuss. >>>> >>>> Best wishes, >>>> Cancai Cai >>>> >>>>> 2026年2月20日 07:25，Mihai Budiu <[email protected]> 写道： >>>>> >>>>> Sure, big files are annoying. >>>>> >>>>> Mihai >>>>> >>>>> From: Stamatis Zampetakis <[email protected]> >>>>> Sent: Thursday, February 19, 2026 5:53 AM >>>>> To: [email protected] <[email protected]> >>>>> Subject: [DISCUSS] Stop testing rules inside RelOptRulesTest >>>>> >>>>> Hi all, >>>>> >>>>> The RelOptRulesTest.java has 12K lines of code and the respective >>>>> RelOptRulesTest.xml file has 22K. Wherever we want to add tests for a >>>>> specific rule we tend to put it there so inevitably these files are >>>>> getting bigger and bigger, navigation becomes harder, conflicts >>>>> increase, naming becomes challenging, etc. >>>>> >>>>> From now onwards we could adopt another pattern where the tests for >>>>> each new rule (e.g., AggregateFilterMagicRule.java) reside into their >>>>> own dedicated testing class (AggregateFilterMagicRuleTest.java) and >>>>> XML file (AggregateFilterMagicRuleTest.xml). In other words, we stop >>>>> adding tests in RelOptRulesTest.java at least for new rules. Thanks to >>>>> the presence of the RelOptFixture class separating tests into >>>>> dedicated classes is trivial and boilerplate code that is required for >>>>> setting it up (e.g., DiffRepo) is rather minimal. >>>>> >>>>> Let me know your thoughts. >>>>> >>>>> Best, >>>>> Stamatis >>>> >>>> >>>
