[
https://issues.apache.org/jira/browse/CALCITE-6335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17944823#comment-17944823
]
Julian Hyde commented on CALCITE-6335:
--------------------------------------
We *could* "run *all* optimization rules with *all* quidem tests and compare
the outputs" as you say, but I'm not convinced we {*}should{*}. As you increase
number of tests, at some point recall is pretty good (you're finding most of
the bugs) and then precision starts decreasing (you're hitting false
positives), and your test suite takes more effort to maintain.
Plus you just can't have a test suite that is giving 200 failures each time you
run it, due to bugs logged but not yet fixed. If the scripts are reused, it
gets difficult to disable a test in scenarios a and c but not in scenarios b
and d.
> Quidem tests should allow specifying optimization passes to apply to programs
> -----------------------------------------------------------------------------
>
> Key: CALCITE-6335
> URL: https://issues.apache.org/jira/browse/CALCITE-6335
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Affects Versions: 1.36.0
> Reporter: Mihai Budiu
> Assignee: Zhen Chen
> Priority: Minor
> Labels: pull-request-available
>
> This facility would make it much easier to test various optimization passes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
