rubenada commented on a change in pull request #1020: [CALCITE-2812] Add algebraic operators to allow expressing recursive queries (Ruben Quesada Lopez) URL: https://github.com/apache/calcite/pull/1020#discussion_r272057767
########## File path: core/src/main/java/org/apache/calcite/adapter/enumerable/EnumerableRecursiveUnion.java ########## @@ -0,0 +1,137 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to you under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.calcite.adapter.enumerable; + +import org.apache.calcite.linq4j.tree.BlockBuilder; +import org.apache.calcite.linq4j.tree.DeclarationStatement; +import org.apache.calcite.linq4j.tree.Expression; +import org.apache.calcite.linq4j.tree.Expressions; +import org.apache.calcite.linq4j.tree.ParameterExpression; +import org.apache.calcite.plan.RelOptCluster; +import org.apache.calcite.plan.RelTraitSet; +import org.apache.calcite.rel.RelNode; +import org.apache.calcite.rel.RelWriter; +import org.apache.calcite.rel.core.SetOp; +import org.apache.calcite.rel.core.Union; +import org.apache.calcite.schema.Table; +import org.apache.calcite.schema.impl.TransientTable; +import org.apache.calcite.util.BuiltInMethod; + +import java.util.List; + + +/** + * Implementation of Recursive Union in + * {@link EnumerableConvention enumerable calling convention}. + */ +public class EnumerableRecursiveUnion extends Union implements EnumerableRel { Review comment: @zabetak Thanks for your feedback. I'll propose the following re-design to implement the "recursive union all" with re-usable modules: Firstly, Create a new `abstract class Iterate extends BiRel` (or perhaps `Iteration` or `IterativeRel` ?) with its corresponding logical / physical operators `LogicalIterate` and `EnumerableIterate`. This Iterate will have two inputs: - The left part will be used as `seed`, and will be evaluated once - The right part will be used as `iteration`, and will be re-evaluated over and over until no results are produced (or until an optional `maxIter` value is reached) This design will allow us to use the `Iterate` as main implementation module for the recursive union (which requires a non-recursive term, executed once; and a recursive term, executed over and over until no results are returned). If, in the future, this `Iterate` needs to be used for other purposes that require just a "pure" iteration (no seed needed), it will still be possible to used it: we will just need to create an Iterate with an empty Values as left (i.e. as seed). Secondly, to implement the data transfer from the recursive union into the "work table", a new, simple `Spool` operator will be created (see CALCITE-481). For the sake of this PR, a naive, lazy spool implementation will be enough. This operator will store in a temporary collection the values returned by its input, before propagating them. When the input returns no more values, the `Spool` will flush its temporary collection into another one (which in the case of the recursive union will be the same collection that will be used to hold the work table). Finally, the temporary table will be a scan on table represented by a list (which will be re-initialized by the `Spool`). To sum up, to represent a recursive union we would have an `Iterate`, with two `Spool`s inserted on top of its left (seed) and right (iteration). To show an example, a typical recursive union like (return numbers from 1 to 10): ``` WITH RECURSIVE delta(n) AS ( VALUES (1) UNION ALL SELECT n+1 FROM delta WHERE n < 10 ) SELECT * FROM delta ``` would be represented with the pseudo-plan: ``` Iterate Spool(table=delta) -- seed, executed once Values({1}) Spool(table=delta) -- iteration, executed until no results are returned Project($0=$0+1) Filter($0<10) Scan(table=delta) ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
