I personally agree with Haisheng Yuan and Rui Wang. Expression normalization is a common technique in many database products and benefits to query optimizer. We should not stop the efforts in this direction. But in current calcite code base, even mininal changes in plan may bring great impacts, considerations on backward compatibility is necessary. Adding optimizer config is a reasonable idea. Because every coin has two sides, for example, some optimizations sacrifice readability when the RelNode/RexNode be converted back to SQL again.
Maybe we can go a step further to provide configuration interfaces, which enable users to determine whether to enable a specified optimization. In frameworks like Hadoop/Hive/Spark/Flink, users can control nearly all of the optimizations through configurations. Best, DonnyZone Danny Chan <[email protected]> 于2019年12月31日周二 上午11:22写道： > Or better, we can make the digest change transparent to user, keep it as > what user writes them, but, in the Planner cache, we use the normalized > digest as the key. > > Best, > Danny Chan > 在 2019年12月31日 +0800 AM9:50，Haisheng Yuan <[email protected]>，写道： > > Normalizing scalar expressions is helpful, especially for deduplicating > derived constraints that is pushed down to child relations. We should not > stop making improvements just because there are many plan changes. > > > > How about adding optimizer config option to enable/disable the feature? > > > > Normalizing every binary operator might seem not necessary, but simply > normalizing equal expression with input refs or literals alone can still > help a lot, I guess, because many of them are generated from equivalent > classes. > > > > - Haisheng > > > > ------------------------------------------------------------------ > > 发件人：Rui Wang<[email protected]> > > 日 期：2019年12月31日 07:09:44 > > 收件人：<[email protected]> > > 主 题：Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form > > > > I think the concern of breaking plan comparison has appeared more than > > once. Not only to this proposal but also to others (e.g. replace "$" with > > "_" in names). > > > > From a think of another perspective, the widely used practice of > > string comparison based plan evaluation also reduces the flexibility of > > making changes in Calcite. We need to carefully think about if a change > has > > a large impact that affects toString(), or something similar. > > > > So if Calcite community could offer another solution for downstream > > projects, and announce that don't guarantee the backward compatibility of > > toString(), then it could be in a better situation. > > > > A few candidate solutions: > > 1. offer utils that convert a plan represented in string to Relnodes. > > 2. use RelBuilder to build plans and does comparison. > > > > and then just maintain the backward compatibility of the solution. > > > > just my two cents. > > > > > > -Rui > > > > > > On Mon, Dec 30, 2019 at 12:20 PM Vladimir Sitnikov < > > [email protected]> wrote: > > > > > The change improves slow tests from 80 min to 60, and the changes are > > > minimal > > > > > > Vladimir > > > > > >
