Thanks a lot Gian. I couldn't grasp your reply then, but after some tracing
through the Druid adapter, I understood what you said about overriding
*deriveRowType,* returning a valid *rowType* and the sketchy *setProjection*
method. What's still bothering me is why does DruidQuery avoid having
inputs and stores them in (*List<RelNode> rels*) instead. I think I'll
start another thread for that.
Julian, no it has nothing to do with nullability difference between
columns. Would you please clarify more about that ?
Thanks,
Gelbana
On Fri, Mar 9, 2018 at 12:32 AM, Julian Hyde <jh...@apache.org> wrote:
> Is it the case that the only difference is nullability of certain columns?
>
> > On Mar 8, 2018, at 10:36 AM, Gian Merlino <g...@imply.io> wrote:
> >
> > Hi Gelbana,
> >
> > My understanding is that the planner rules are not meant to change row
> > types or query behavior. They are just meant to transform into something
> > that is equivalent, but hopefully better in some way (lower cost, proper
> > calling convention, etc).
> >
> > What you can do is have your SQLiQuery's deriveRowType method return
> > something different if you have pushed a projection into it. If the
> > SQLiQuery is going to do the projection, then it should return a row type
> > that matches the projection.
> >
> > Also having "setProjection" method looks sketchy to me. Presumably that
> > should change the digest of the SQLiQuery, but I am not sure if anything
> is
> > allowed to do that except replaceInput?
> >
> > Gian
> >
> > On Thu, Mar 8, 2018 at 9:27 AM, Muhammad Gelbana <m.gelb...@gmail.com>
> > wrote:
> >
> >> I'm trying to push down projection to a bindable node but I'm facing the
> >> following error.
> >>
> >> java.lang.AssertionError: Type mismatch:
> >> rowtype of new rel:
> >> RecordType(INTEGER NOT NULL EXPR$0) NOT NULL
> >> rowtype of set:
> >> RecordType(VARCHAR CHARACTER SET "ISO-8859-1" COLLATE
> >> "ISO-8859-1$en_US$primary" NOT NULL COUNTRY_ISO_CODE, VARCHAR CHARACTER
> SET
> >> "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary" NOT NULL COUNTRY_NAME,
> >> VARCHAR CHARACTER SET "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary"
> NOT
> >> NULL COUNTRY_SUBREGION, VARCHAR CHARACTER SET "ISO-8859-1" COLLATE
> >> "ISO-8859-1$en_US$primary" NOT NULL COUNTRY_REGION, VARCHAR CHARACTER
> SET
> >> "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary" NOT NULL COUNTRY_TOTAL,
> >> VARCHAR CHARACTER SET "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary"
> NOT
> >> NULL COUNTRY_NAME_HIST, VARCHAR CHARACTER SET "ISO-8859-1" COLLATE
> >> "ISO-8859-1$en_US$primary" NOT NULL DEFAULT_KW, VARCHAR CHARACTER SET
> >> "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary" NOT NULL
> CURRENT_TIMESTAMP,
> >> VARCHAR CHARACTER SET "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary"
> NOT
> >> NULL ANALYSE) NOT NULL
> >> at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31)
> >> at org.apache.calcite.plan.RelOptUtil.equal(RelOptUtil.java:1864)
> >> at org.apache.calcite.plan.volcano.RelSubset.add(RelSubset.java:271)
> >> at org.apache.calcite.plan.volcano.RelSet.add(RelSet.java:148)
> >> at
> >> org.apache.calcite.plan.volcano.VolcanoPlanner.
> addRelToSet(VolcanoPlanner.
> >> java:1644)
> >> at
> >> org.apache.calcite.plan.volcano.VolcanoPlanner.
> reregister(VolcanoPlanner.
> >> java:1319)
> >> at org.apache.calcite.plan.volcano.RelSet.mergeWith(RelSet.java:331)
> >> at
> >> org.apache.calcite.plan.volcano.VolcanoPlanner.merge(
> >> VolcanoPlanner.java:1410)
> >> at
> >> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(
> >> VolcanoPlanner.java:878)
> >> at
> >> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(
> >> VolcanoPlanner.java:1766)
> >> at
> >> org.apache.calcite.plan.volcano.VolcanoRuleCall.
> >> transformTo(VolcanoRuleCall.java:135)
> >> at
> >> org.apache.calcite.plan.RelOptRuleCall.transformTo(
> >> RelOptRuleCall.java:234)
> >> at
> >> com.sqlinterface.data.provider.calcite.internal.
> >> SQLiProjectionRule.onMatch(SQLiProjectionRule.java:24)
> >> at
> >> org.apache.calcite.plan.volcano.VolcanoRuleCall.
> >> onMatch(VolcanoRuleCall.java:212)
> >> at
> >> org.apache.calcite.plan.volcano.VolcanoPlanner.
> findBestExp(VolcanoPlanner.
> >> java:650)
> >> at org.apache.calcite.tools.Programs$5.run(Programs.java:326)
> >> at
> >> org.apache.calcite.tools.Programs$SequenceProgram.run(
> Programs.java:387)
> >> at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:188)
> >> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:319)
> >> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:230)
> >> at
> >> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(
> >> CalcitePrepareImpl.java:781)
> >> at
> >> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(
> >> CalcitePrepareImpl.java:640)
> >> at
> >> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(
> >> CalcitePrepareImpl.java:610)
> >> at
> >> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(
> >> CalciteConnectionImpl.java:221)
> >> at
> >> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(
> >> CalciteMetaImpl.java:603)
> >> at
> >> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(
> >> AvaticaConnection.java:638)
> >> at
> >> org.apache.calcite.avatica.AvaticaStatement.executeInternal(
> >> AvaticaStatement.java:149)
> >> at
> >> org.apache.calcite.avatica.AvaticaStatement.executeQuery(
> >> AvaticaStatement.java:218)
> >> at
> >> com.sqlinterface.data.provider.calcite.internal.
> >> Test_SQLi.test(Test_SQLi.java:32)
> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> at
> >> sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:
> >> 62)
> >> at
> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> >> DelegatingMethodAccessorImpl.java:43)
> >> at java.lang.reflect.Method.invoke(Method.java:498)
> >> at
> >> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
> >> FrameworkMethod.java:50)
> >> at
> >> org.junit.internal.runners.model.ReflectiveCallable.run(
> >> ReflectiveCallable.java:12)
> >> at
> >> org.junit.runners.model.FrameworkMethod.invokeExplosively(
> >> FrameworkMethod.java:47)
> >> at
> >> org.junit.internal.runners.statements.InvokeMethod.
> >> evaluate(InvokeMethod.java:17)
> >> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> >> at
> >> org.junit.runners.BlockJUnit4ClassRunner.runChild(
> >> BlockJUnit4ClassRunner.java:78)
> >> at
> >> org.junit.runners.BlockJUnit4ClassRunner.runChild(
> >> BlockJUnit4ClassRunner.java:57)
> >> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> >> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> >> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> >> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> >> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> >> at
> >> org.junit.internal.runners.statements.RunBefores.
> >> evaluate(RunBefores.java:26)
> >> at
> >> org.junit.internal.runners.statements.RunAfters.evaluate(
> >> RunAfters.java:27)
> >> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> >> at
> >> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(
> >> JUnit4TestReference.java:86)
> >> at
> >> org.eclipse.jdt.internal.junit.runner.TestExecution.
> >> run(TestExecution.java:38)
> >> at
> >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.
> >> runTests(RemoteTestRunner.java:459)
> >> at
> >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.
> >> runTests(RemoteTestRunner.java:678)
> >> at
> >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.
> >> run(RemoteTestRunner.java:382)
> >> at
> >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.
> >> main(RemoteTestRunner.java:192)
> >>
> >> Here is the rule I'm using
> >>
> >> public class SQLiProjectionRule extends RelOptRule {
> >> public SQLiProjectionRule() {
> >> super(operand(LogicalProject.class, operand(SQLiQuery.class,
> >> any())), "SQLiProjectionRule");
> >> }
> >> @Override
> >> public Convention getOutConvention() {
> >> return BindableConvention.INSTANCE;
> >> }
> >> @Override
> >> public void onMatch(RelOptRuleCall call) {
> >> SQLiQuery query = call.rel(1);
> >> query.setProjection(call.rel(0)); //Now the SQLiQuery node knows
> >> about the projected columns
> >> call.transformTo(query); // Replace LogicalAggregate with
> >> SQLiQuery..but it fails as mentioned.
> >> }
> >> }
> >>
> >> What I'm trying to do is to replace the *LogicalProject* node with my
> >> *SQLiQuery* node that includes the table scanning information and
> projected
> >> columns information too after the rule is fired.
> >>
> >> Is this an invalid approach ? What is the correct way to achieve my goal
> >> please ?
> >>
> >> Thanks,
> >> Gelbana
> >>
>
>
