Hi Krishnakant, Why do you need to maintain the structure of RexSubquery? Most of the systems relying on Calcide do use the SubQueryRemoveRule so I am curious why you cannot use it in your system.
Best, Stamatis On Mon, Jun 14, 2021 at 10:44 AM Krishnakant Agrawal <[email protected]> wrote: > Haisheng, No I included just 4-5 rules which I don't think they tinker with > subqueries. > I found SubQueryRemoveRule with three configs. > Are there any others too? Please let us know. > I don't want to include this rule because I want to maintain these > structures post-optimization too. > > Julian, Yes I was referring to scalar subqueries. > The subqueries if they are in a Rex Form are not picked up for optimization > by default. > If someone from the core team wants to expand on this please do, I might be > wrong. > I support the above statement because the Rel Inside the RexSubQuery was > NOT optimized. > But with the same set of rules, it got optimized when sent to HepPlanner as > the main rel. > > As of now, I will be writing a new Rule which basically does exactly what > Julian said of scanning Rexes for RexSubquery > and applying rules on them. > > This rule will need to be added to the Rule List so it's not enabled by > default. > Please let me know a more graceful/optimized way to do this if any. > > Sample RexSubquery(IN operator):- > > LogicalFilter(condition=[IN($8, { > LogicalProject(EXPR$0=[$8]) > LogicalFilter(condition=[=($7, 1)]) > LogicalTableScan(table=[[foodmart, employee]]) > })]) > LogicalTableScan(table=[[foodmart, employee]]) > > > On Thu, Jun 10, 2021 at 12:43 AM Julian Hyde <[email protected]> > wrote: > > > I presume we’re talking about scalar sub-queries (e.g. sub-query in > > SELECT) and sub-queries with IN and EXISTS, as opposed to sub-queries in > > the FROM clause. The former are represented using RexSubQuery because the > > queries appear in scalar expressions. > > > > There are rules to expand RexSubQuery into relational operators. After > > those rules have been applied, then I’m sure that the Hep engine will be > > able to find them. I think that’s what Haisheng is referring to. > > > > But while those sub-queries are still wrapped in RexSubQuery, it’s > > possible that the Hep engine won’t find them. Because it will have to > walk > > over every RexNode inside a Project, Filter or Join and if that RexNode > is > > a RexSubQuery, traverse into it to apply the rules to the RelNodes > inside. > > I don’t know whether that happens today. Someone should check. It’s a > > reasonable ask (but there might be unforeseen consequences if we enable > it > > by default). > > > > Julian > > > > > > > > > On Jun 9, 2021, at 12:02 PM, Haisheng Yuan <[email protected]> wrote: > > > > > > Did you include the subquery related rules in the HepPlanner? > > > > > > Haisheng > > > > > > On 2021/06/09 17:59:44, Krishnakant Agrawal <[email protected]> > > wrote: > > >> Hi All, > > >> > > >> When running a HepOptimizer on top of a RelNode which has a subquery > > >> embedded in it, The Optimizer does not take the RelNode representing > the > > >> subquery up for optimization. > > >> > > >> Is this by design, what's the correct way for the Subquery RelNode to > > be > > >> picked up for Optimization? > > >> > > >> If this is a bug, what is the conceptually correct way to go about > > fixing > > >> this? > > >> > > >> Thanks, > > >> KK > > >> > > > > >
