Hi P.F.ZHAN,in calcite,it has a method RexUtil#expandSearch to expand Search,maybe you could get some information from this method.
There is also some logic to simplify Search in the RexSimplify#simplifySearch method. I hope this could help you. Here's the code: 1. https://github.com/apache/calcite/blob/98f3048fb1407e2878162ffc80388d4f9dd094b2/core/src/main/java/org/apache/calcite/rex/RexUtil.java#L593 2. https://github.com/apache/calcite/blob/98f3048fb1407e2878162ffc80388d4f9dd094b2/core/src/main/java/org/apache/calcite/rex/RexSimplify.java#L2132 Best, LakeShen Soumyadeep Mukhopadhyay <[email protected]> 于2023年8月4日周五 20:29写道： > Thank you, shall explore more on this! :) > > > On Fri, 4 Aug 2023 at 5:53 PM, P.F. ZHAN <[email protected]> wrote: > > > Aha, I'm using Apache Kylin which uses Calcite to generate a logical > plan, > > then convert to Spark plan to execute a query. Given that Calcite has > more > > operations for aggregations, and Kylin wants to take full advantage of > > precomputed cubes (something like Calcite's materialized views), it uses > > both Calcite and Spark(for distribution computing). Maybe it's wild and a > > little fun, but it does works well on many scenarios. > > > > On Fri, Aug 4, 2023 at 8:10 PM Soumyadeep Mukhopadhyay < > > [email protected]> wrote: > > > > > I am curious about your use case. Are you not losing out on the > > > optimisations of Calcite when you are using Spark? Is it possible for > you > > > to share a general approach where we will be able to keep the > > optimisations > > > done by Calcite and use Spark on top of it? > > > > > > > > > On Fri, 4 Aug 2023 at 5:19 PM, P.F. ZHAN <[email protected]> wrote: > > > > > > > Generally speaking, the SEARCH operator is very good, but when we use > > > > Calcite to optimize the logical plan and then use Spark to execute, > > this > > > is > > > > unsupported. So is there a more elegant way to close the SEARCH > > operator? > > > > Or how to convert the SEARCH operator to the IN operator before > > > converting > > > > the Calcite logical plan to the Spark logical plan? If we do this, we > > > need > > > > to consider Join / Filter, are there any other RelNodes? > > > > > > > > Maybe, this optimization is optional more better at present for many > > > query > > > > execution engine does not support this operator? > > > > > > > > > >
