The plan seems reasonable to me so as you said there seems to be a problem with the Interpreter. Overriding childrenAccept seems like a good workaround but I think it would be nice to fix the Interpreter. Can you please log a JIRA issue for this?
As a general remark, I don't use Bindable convention very often since it is not very well tested. Although Enumerable is a bit more difficult to understand it is more stable. Maybe it would be a good idea to change your converter from Bindable to Enumerable. Best, Stamatis On Mon, May 6, 2019 at 5:13 PM Muhammad Gelbana <[email protected]> wrote: > Thank you all for your time but unfortunately I'm unable to overcome this > challenge so far. Allow me to explain more. > > *This is how I'm using calcite* > Planner planner = Frameworks.getPlanner(frameworkConfig); // > frameworkConfig is irrelevant I suppose > SqlNode parsed = planner.parse("SELECT Tenant FROM Audit.audit where Tenant > != 'castle'"); > SqlNode validated = planner.validate(parsed); > RelRoot root = planner.rel(validated); > PreparedStatement prepared = RelRunners.run(root.rel); > ResultSet rs = prepared.executeQuery()); > > *This is the final physical plan I'm achieving* > EnumerableInterpreter: rowcount = 50.0, cumulative cost = {25.0 rows, 25.0 > cpu, 0.0 io}, id = 102 > BindableProject(Tenant=[$2]): rowcount = 50.0, cumulative cost = {160.0 > rows, 166.0 cpu, 0.0 io}, id = 100 > GelbanaBindableRel: rowcount = 50.0, cumulative cost = {110.0 rows, > 116.0 cpu, 0.0 io}, id = 98 *// My converter node* > GelbanaFilter(condition=[<>($2, 'demo')]): rowcount = 50.0, > cumulative cost = {105.0 rows, 111.0 cpu, 0.0 io}, id = 96 > GelbanaQuery(table=[[Audit, audit]]): rowcount = 100.0, cumulative > cost = {100.0 rows, 101.0 cpu, 0.0 io}, id = 3 *// My table scan* > > My problem is that the interpreter goes through all the plan's operators > [1] and for each operator it attempts to wrap it as an instance of > org.apache.calcite.interpreter.Node [2] so the interpreter would call the > Node's method "run" [3] for each wrapped operator when it starts executing > the plan. But if it couldn't wrap the operator as a Node [4], it checks if > the operator implements the InterpretableRel interface to call it's > "implement" method. But if it doesn't implement the InterpretableRel > interface in that case, an exception is thrown. > > My issue is that I don't need my Gelbana operators, including the table > scan which is this thread is all about, to be wrapped as nodes neither > implement the InterpretableRel interface because my converter > (GelbanaBindableRel) will be responsible for running its inputs, including > eventually the table scan(s). > > To fix that, I did the following for my converter > @Override > public void childrenAccept(RelVisitor visitor) { > if (visitor instanceof CoreCompiler) return; > } > > This allows me to prevent the interpreter from visiting my converter's > inputs [1] to avoid the trouble mentioned above. But it feels like I'm > hacking my way through calcite and not using it the way its desgined to. Or > is this a correct approach ? Could this have any side effects I might worry > about ? > > [1] > > https://github.com/apache/calcite/blob/0d504d20d47542e8d461982512ae0e7a94e4d6cb/core/src/main/java/org/apache/calcite/interpreter/Interpreter.java#L447 > [2] > > https://github.com/apache/calcite/blob/0d504d20d47542e8d461982512ae0e7a94e4d6cb/core/src/main/java/org/apache/calcite/interpreter/Interpreter.java#L451 > [3] > > https://github.com/apache/calcite/blob/0d504d20d47542e8d461982512ae0e7a94e4d6cb/core/src/main/java/org/apache/calcite/interpreter/Interpreter.java#L130 > [4] > > https://github.com/apache/calcite/blob/0d504d20d47542e8d461982512ae0e7a94e4d6cb/core/src/main/java/org/apache/calcite/interpreter/Interpreter.java#L452 > > Thanks, > Gelbana > > > On Sun, May 5, 2019 at 11:43 PM Stamatis Zampetakis <[email protected]> > wrote: > > > Hi Muhammad, > > > > I'm not sure why you need to implement InterpretableRel interface. I'm > > probably missing some details. > > > > I suppose you are using Calcite through the JDBC interface thus I guess > you > > are relying on CalcitePrepareImpl. > > If that's the case then using TranslatableTable interface seems like a > good > > idea. > > The latter means that you are using the default VolcanoPlanner that > expects > > the final result to be in Bindable or Enumerable convention [2]. > > You may define various operators and tables to be in CustomConvention but > > for sure you need a Converter > > and respective rules to go from CustomConvention to EnumerableConvention > > (similar to MongoToEnumerableConverter and > MongoToEnumerableConverterRule). > > > > The final plan should look be similar to the one below: > > > > CustomToEnumerableConverter > > CustomProject(product_id=[CAST(ITEM($0, 'product_id')):DOUBLE]) > > CustomTableScan(table=[[_foodmart, sales_fact_1998]])" > > > > Best, > > Stamatis > > > > [2] > > > > > https://github.com/apache/calcite/blob/0d504d20d47542e8d461982512ae0e7a94e4d6cb/core/src/main/java/org/apache/calcite/prepare/CalcitePrepareImpl.java#L719 > > > > On Sun, May 5, 2019 at 4:13 AM Yuzhao Chen <[email protected]> wrote: > > > > > You can set up the target traits to include the convention you want > when > > > invoke the Program#run method [1], with the converters as rules of the > > > program. > > > > > > [1] > > > > > > https://github.com/apache/calcite/blob/9ece70f5dcdb00dbc6712496c51f52c05178d4aa/core/src/main/java/org/apache/calcite/tools/Program.java#L38 > > > > > > Best, > > > Danny Chan > > > 在 2019年5月4日 +0800 PM6:48，Muhammad Gelbana <[email protected]>，写道： > > > > I implemented a new convention but I'm facing difficulties with > > > > implementing tables for that convention. > > > > > > > > Since I need to apply my convention rules and converters, I assume I > > must > > > > implement TranslatableTable so I can override the RelNode produced by > > the > > > > toRel method and provide the rules I need through the > > > > RelNode.register(RelOptPlanner planner) call back (Is there another > way > > > ?) > > > > > > > > At the same time, I need for my table's RelNode to implement my > > > > convention's interface. > > > > > > > > After implementing TranslatableTable and my convention's interface, I > > > > discovered that my table's RelNode have to either implement an > > > interpreter > > > > node or the InterpretableRel interface. But this led to what seems to > > me > > > as > > > > that my table scan is executed twice. > > > > > > > > I tried many things, such as having my table extend TableScan, > > > > AbstractEnumerable2 QueryableTable and other interfaces/class that > > either > > > > didn't give me what I want or failed with strange exceptions. > > > > > > > > What I need is to implement my table scan following the same > convention > > > I'm > > > > writing. So what is the correct way to do this ? > > > > > > > > The MongoDB adapter way is to implement the TranslatableTable and the > > > > QueryableTable interfaces, but that's confusing me. I mean, it's > > already > > > > implementing its own new convention interface, why would it need to > > > > implement another one to implement its table scans ? I thought while > > > > executing the final plan, the first calcite identified node > (enumerable > > > or > > > > bindable, enumerable as in MongoDB adapter's example), the > interpreter > > > > would execute that node, which will internally execute all its input > > > nodes > > > > including the table scan, correct ? > > > > > > > > Thanks, > > > > Gelbana > > > > > >
