@Haisheng > Calcite uses Project operator and all kinds of ProjectXXXTranposeRule to > prune unused columns.
I also noticed that in most cases Project-related rules took significant part of the planning time. But I didn't explore this problem yet. > MEMO group (RelSet) represents logically equivalent expressions. All the > expressions in one group should share the same logical properties, e.g. > functional dependency, constraint properties etc. But they are not sharing > it. Computation is done for each individual operator. I thought the equivalence of logical properties within groups (RelSets) are implicit. For example, in RelSet#addInternal it is always verified that the new added node has the same type as other members of the set. Anyway I absolutely agree with you that problems with traits propagation, rules matching and other that you mentioned in the previous e-mails should be solved in the first place. We need first to make Volcano optimizer work right and only then make some improvements like search space pruning. I would love to join this work to improve Volcano planner. Looking forward for design doc. -- Kind Regards Roman Kondakov On 11.01.2020 11:42, Haisheng Yuan wrote: > Currently, Calcite uses Project operator and all kinds of > ProjectXXXTranposeRule to prune unused columns. Every operator's output > columns use index to reference child operators' columns. If there is a > Project operator with child operator of a Filter, if we push project down > under Filter, we will have Project-Filter-Project-FilterInput. All the newly > generated relnodes will trigger rule matches. e.g. If we already did > ReduceExpressionRule on filter, but due to the new filter RexCall's input ref > index changed, we have to apply ReduceExpressionRule on the new filter again, > even there is nothing can be reduced. Similarly new operator transpose/merge > rule will be triggered. This can trigger a lot of rule matches. > > MEMO group (RelSet) represents logically equivalent expressions. All the > expressions in one group should share the same logical properties, e.g. > functional dependency, constraint properties etc. But they are not sharing > it. Computation is done for each individual operator. > > Without resolving those issue, space pruning won't help much. > > There are a lot of room for improvement. Hope the community can join the > effort to make Calcite better. > > - Haisheng > > ------------------------------------------------------------------ > 发件人：Roman Kondakov<kondako...@mail.ru.INVALID> > 日 期：2020年01月10日 19:39:51 > 收件人：<dev@calcite.apache.org> > 主 题：Re: [DISCUSS] Proposal to add API to force rules matching specific rels > > @Haisheng, could you please clarify what you mean by these points? > >> - the poor-design of column pruning, >> - lack of group properties etc. > > I guess I'm not aware of these problems. >
