[jira] [Comment Edited] (CALCITE-2170) Use Druid Expressions capabilities to improve the amount of work that can be pushed to Druid
[
https://issues.apache.org/jira/browse/CALCITE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368089#comment-16368089
]
Jesus Camacho Rodriguez edited comment on CALCITE-2170 at 2/17/18 3:36 AM:
---
Fixed in http://git-wip-us.apache.org/repos/asf/calcite/commit/0e13a8a , thanks
[~bslim].
was (Author: jcamachorodriguez):
Fixed in http://git-wip-us.apache.org/repos/asf/calcite/commit/5acf84f , thanks
[~bslim].
> Use Druid Expressions capabilities to improve the amount of work that can be
> pushed to Druid
>
>
> Key: CALCITE-2170
> URL: https://issues.apache.org/jira/browse/CALCITE-2170
> Project: Calcite
> Issue Type: New Feature
> Components: druid
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Fix For: 1.16.0
>
>
> Druid 0.11 has newly built in capabilities called Expressions that can be
> used to push expression like projects/aggregates/filters.
> In order to leverage this new feature, some changes need to be done to the
> Druid Calcite adapter.
> This is a link to the current supported functions and expressions in Druid
> [http://druid.io/docs/latest/misc/math-expr.html]
> As you can see from the Docs an expression can be an actual tree of operators,
> Expression can be used with Filters, Projects, Aggregates, PostAggregates and
> Having filters. For Filters will have new Filter kind called Filter
> expression.
> FYI, you might ask can we push everything as Expression Filter the short
> answer
> is no because, other kinds of Druid filters perform better when used, Hence
> Expression filter is a plan B sort of thing. In order to push expression as
> Projects and Aggregates we will be using Expression based Virtual Columns.
> The major change is the merging of the logic of pushdown verification code and
> the Translation of RexCall/RexNode to Druid Json, native physical language.
> The
> main drive behind this redesign is the fact that in order to check if we can
> push down a tree of expressions to Druid we have to compute the Druid
> Expression
> String anyway. Thus instead of having 2 different code paths, one for pushdown
> validation and one for Json generation we can have one function that does
> both.
> For instance instead of having one code path to test and check if a given
> filter
> can be pushed or not and then having a translation layer code, will have
> one function that either returns a valid Druid Filter or null if it is not
> possible to pushdown. The same idea will be applied to how we push Projects
> and
> Aggregates, Post Aggregates and Sort.
> Here are the main elements/Classes of the new design. First will be merging
> the logic of
> Translation of Literals/InputRex/RexCall to a Druid physical representation.
> Translate leaf RexNode to Valid pair Druid Column + Extraction functions if
> possible
> {code:java}
> /**
> * @param rexNode leaf Input Ref to Druid Column
> * @param rowType row type
> * @param druidQuery druid query
> *
> * @return {@link Pair} of Column name and Extraction Function on the top of
> the input ref or
> * {@link Pair of(null, null)} when can not translate to valid Druid column
> */
> protected static Pair toDruidColumn(RexNode
> rexNode,
> RelDataType rowType, DruidQuery druidQuery
> )
> {code}
> In the other hand, in order to Convert Literals to Druid Literals will
> introduce
> {code:java}
> /**
> * @param rexNode rexNode to translate to Druid literal equivalante
> * @param rowType rowType associated to rexNode
> * @param druidQuery druid Query
> *
> * @return non null string or null if it can not translate to valid Druid
> equivalent
> */
> @Nullable
> private static String toDruidLiteral(RexNode rexNode, RelDataType rowType,
> DruidQuery druidQuery
> )
> {code}
> Main new functions used to pushdown nodes and Druid Json generation.
> Filter pushdown verification and generates is done via
> {code:java}
> org.apache.calcite.adapter.druid.DruidJsonFilter#toDruidFilters
> {code}
> For project pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeProjectAsScan.
> {code}
> For Grouping pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeProjectGroupSet.
> {code}
> For Aggregation pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeDruidJsonAgg
> {code}
> For sort pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeSort\{code}
> Pushing of PostAggregates will be using Expression post Aggregates and use
> {code}
> org.apache.calcite.adapter.druid.DruidExpressions#toDruidExpression\{code}
> to generate expression
> For Expression computation most of
[jira] [Resolved] (CALCITE-2170) Use Druid Expressions capabilities to improve the amount of work that can be pushed to Druid
[
https://issues.apache.org/jira/browse/CALCITE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-2170.
--
Resolution: Fixed
Fix Version/s: 1.16.0
Fixed in http://git-wip-us.apache.org/repos/asf/calcite/commit/5acf84f , thanks
[~bslim].
> Use Druid Expressions capabilities to improve the amount of work that can be
> pushed to Druid
>
>
> Key: CALCITE-2170
> URL: https://issues.apache.org/jira/browse/CALCITE-2170
> Project: Calcite
> Issue Type: New Feature
> Components: druid
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Fix For: 1.16.0
>
>
> Druid 0.11 has newly built in capabilities called Expressions that can be
> used to push expression like projects/aggregates/filters.
> In order to leverage this new feature, some changes need to be done to the
> Druid Calcite adapter.
> This is a link to the current supported functions and expressions in Druid
> [http://druid.io/docs/latest/misc/math-expr.html]
> As you can see from the Docs an expression can be an actual tree of operators,
> Expression can be used with Filters, Projects, Aggregates, PostAggregates and
> Having filters. For Filters will have new Filter kind called Filter
> expression.
> FYI, you might ask can we push everything as Expression Filter the short
> answer
> is no because, other kinds of Druid filters perform better when used, Hence
> Expression filter is a plan B sort of thing. In order to push expression as
> Projects and Aggregates we will be using Expression based Virtual Columns.
> The major change is the merging of the logic of pushdown verification code and
> the Translation of RexCall/RexNode to Druid Json, native physical language.
> The
> main drive behind this redesign is the fact that in order to check if we can
> push down a tree of expressions to Druid we have to compute the Druid
> Expression
> String anyway. Thus instead of having 2 different code paths, one for pushdown
> validation and one for Json generation we can have one function that does
> both.
> For instance instead of having one code path to test and check if a given
> filter
> can be pushed or not and then having a translation layer code, will have
> one function that either returns a valid Druid Filter or null if it is not
> possible to pushdown. The same idea will be applied to how we push Projects
> and
> Aggregates, Post Aggregates and Sort.
> Here are the main elements/Classes of the new design. First will be merging
> the logic of
> Translation of Literals/InputRex/RexCall to a Druid physical representation.
> Translate leaf RexNode to Valid pair Druid Column + Extraction functions if
> possible
> {code:java}
> /**
> * @param rexNode leaf Input Ref to Druid Column
> * @param rowType row type
> * @param druidQuery druid query
> *
> * @return {@link Pair} of Column name and Extraction Function on the top of
> the input ref or
> * {@link Pair of(null, null)} when can not translate to valid Druid column
> */
> protected static Pair toDruidColumn(RexNode
> rexNode,
> RelDataType rowType, DruidQuery druidQuery
> )
> {code}
> In the other hand, in order to Convert Literals to Druid Literals will
> introduce
> {code:java}
> /**
> * @param rexNode rexNode to translate to Druid literal equivalante
> * @param rowType rowType associated to rexNode
> * @param druidQuery druid Query
> *
> * @return non null string or null if it can not translate to valid Druid
> equivalent
> */
> @Nullable
> private static String toDruidLiteral(RexNode rexNode, RelDataType rowType,
> DruidQuery druidQuery
> )
> {code}
> Main new functions used to pushdown nodes and Druid Json generation.
> Filter pushdown verification and generates is done via
> {code:java}
> org.apache.calcite.adapter.druid.DruidJsonFilter#toDruidFilters
> {code}
> For project pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeProjectAsScan.
> {code}
> For Grouping pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeProjectGroupSet.
> {code}
> For Aggregation pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeDruidJsonAgg
> {code}
> For sort pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeSort\{code}
> Pushing of PostAggregates will be using Expression post Aggregates and use
> {code}
> org.apache.calcite.adapter.druid.DruidExpressions#toDruidExpression\{code}
> to generate expression
> For Expression computation most of the work is done here
> {code:java}
> org.apache.calcite.adapter.druid.DruidExpressions#toDruidExpression\{code}
> This static function generates Druid String
[jira] [Commented] (CALCITE-2177) TUMBLE_START does not respect AS when SELECT and GROUP BY match
[
https://issues.apache.org/jira/browse/CALCITE-2177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368004#comment-16368004
]
Julian Hyde commented on CALCITE-2177:
--
Glad this works now. If you run into complex cases, consider contributing tests
to Calcite. Even if the tests work, you are helping to ensure that the
functionality stays working.
> TUMBLE_START does not respect AS when SELECT and GROUP BY match
> ---
>
> Key: CALCITE-2177
> URL: https://issues.apache.org/jira/browse/CALCITE-2177
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.13.0
>Reporter: Andrew Pilloud
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.15.0
>
>
> When the order of SELECT and GROUP BY arguments match, the output of
> TUMBLE_START and HOP_START can not be named with AS. Found while trying to
> implement Nexmark benchmark queries in Apache Beam.
> This query fails with "An exception occured while executing the Java class.
> null: InvocationTargetException: Type 'RecordType(TIMESTAMP(0) $f0)' has no
> field 'starttime'"
> {code:sql}
> SELECT B.starttime FROM
>(SELECT
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
> This more useful query also fails with "An exception occured while executing
> the Java class. null: InvocationTargetException: Type 'RecordType(BIGINT
> auction, BIGINT price, TIMESTAMP(0) $f2)' has no field 'starttime'"
> {code:sql}
> SELECT B.starttime FROM
>(SELECT B1.auction, B1.price,
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY B1.auction, B1.price,
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
> However swap the order of the select arguments and it works as expected:
> {code:sql}
> SELECT B.starttime FROM
>(SELECT B1.price, B1.auction,
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY B1.auction, B1.price,
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
>
> More detailed stack from second query:
> {code:java}
> Caused by: java.lang.AssertionError: Type 'RecordType(TIMESTAMP(0) $f0,
> BIGINT auction, BIGINT price)' has no field 'starttime'
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.rex.RexBuilder.makeFieldAccess(RexBuilder.java:175)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3533)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.access$1800(SqlToRelConverter.java:210)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4545)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3904)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:344)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4438)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:3744)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:662)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:619)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3054)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:555)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.prepare.PlannerImpl.rel(PlannerImpl.java:232)
> at
> org.apache.beam.sdk.extensions.sql.impl.planner.BeamQueryPlanner.convertToRelNode(BeamQueryPlanner.java:164)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-508) Reading from ResultSet before calling next() should throw SQLException not NoSuchElementException
[ https://issues.apache.org/jira/browse/CALCITE-508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368001#comment-16368001 ] ASF GitHub Bot commented on CALCITE-508: Github user julianhyde commented on the issue: https://github.com/apache/calcite-avatica/pull/23 Sorry for the delay. I've made https://issues.apache.org/jira/browse/CALCITE-508 a dependency of the release https://issues.apache.org/jira/browse/CALCITE-2182, so we'll get to it in time. > Reading from ResultSet before calling next() should throw SQLException not > NoSuchElementException > - > > Key: CALCITE-508 > URL: https://issues.apache.org/jira/browse/CALCITE-508 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Labels: newbie > > Reading from ResultSet before calling next() should throw SQLException not > NoSuchElementException. > Each of the Cursor.Accessor.getXxx methods should convert runtime exceptions > to SQLException. > JdbcTest.testExtract currently demonstrates this problem; it passes if there > is a NoSuchElementException, but should look for a SQLException. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-508) Reading from ResultSet before calling next() should throw SQLException not NoSuchElementException
[ https://issues.apache.org/jira/browse/CALCITE-508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde updated CALCITE-508: Component/s: avatica > Reading from ResultSet before calling next() should throw SQLException not > NoSuchElementException > - > > Key: CALCITE-508 > URL: https://issues.apache.org/jira/browse/CALCITE-508 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Labels: newbie > > Reading from ResultSet before calling next() should throw SQLException not > NoSuchElementException. > Each of the Cursor.Accessor.getXxx methods should convert runtime exceptions > to SQLException. > JdbcTest.testExtract currently demonstrates this problem; it passes if there > is a NoSuchElementException, but should look for a SQLException. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-2128) Add jethro sql dialect and jdbc support
[ https://issues.apache.org/jira/browse/CALCITE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde resolved CALCITE-2128. -- Resolution: Fixed Fixed in [facd83d3|http://git-wip-us.apache.org/repos/asf/calcite/commit/facd83d3]; thanks for this contribution, [~msydoron]! > Add jethro sql dialect and jdbc support > --- > > Key: CALCITE-2128 > URL: https://issues.apache.org/jira/browse/CALCITE-2128 > Project: Calcite > Issue Type: Improvement > Components: jdbc-adapter >Reporter: Jonathan Doron >Assignee: Jonathan Doron >Priority: Major > Fix For: 1.16.0 > > > Calcite Jdbc operators code needs to be public so it could be used in hive > code to support usage of external jdbc tables in hive -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2128) In JDBC adapter, add SQL dialect for Jethro Data
[ https://issues.apache.org/jira/browse/CALCITE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde updated CALCITE-2128: - Summary: In JDBC adapter, add SQL dialect for Jethro Data (was: Add jethro sql dialect and jdbc support) > In JDBC adapter, add SQL dialect for Jethro Data > > > Key: CALCITE-2128 > URL: https://issues.apache.org/jira/browse/CALCITE-2128 > Project: Calcite > Issue Type: Improvement > Components: jdbc-adapter >Reporter: Jonathan Doron >Assignee: Jonathan Doron >Priority: Major > Fix For: 1.16.0 > > > Calcite Jdbc operators code needs to be public so it could be used in hive > code to support usage of external jdbc tables in hive -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-2059) Apache Geode adapter
[ https://issues.apache.org/jira/browse/CALCITE-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde resolved CALCITE-2059. -- Resolution: Fixed Fix Version/s: 1.16.0 Fixed in [707f4de9|http://git-wip-us.apache.org/repos/asf/calcite/commit/707f4de9]; many thanks for this major contribution, [~tzolov]! > Apache Geode adapter > > > Key: CALCITE-2059 > URL: https://issues.apache.org/jira/browse/CALCITE-2059 > Project: Calcite > Issue Type: New Feature > Components: geode >Reporter: Christian Tzolov >Assignee: Julian Hyde >Priority: Major > Fix For: 1.16.0 > > > I've been working on a Calcite adapter for [Apache > Geode|http://geode.apache.org]. > Current implementation uses the plain Geode API and > [OQL|http://geode.apache.org/docs/guide/13/developing/querying_basics/chapter_overview.html](Object > Query Interface) to push down relational expressions such as projections, > filtering, sorting, and grouping . > Provided functionality can hopefully address certain Geode use cases and will > provide a stepping stone for future improvements. > Here are some remaining tasks as i see it: > * New tests for test suite (and update calcite-test-dataset to support Geode) > * Add Integration tests that use calcite-test-dataset > * Documentation -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-2180) Invalid code generated for negative of byte and short values
[
https://issues.apache.org/jira/browse/CALCITE-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde resolved CALCITE-2180.
--
Resolution: Fixed
Fix Version/s: 1.16.0
Fixed in
[3c67a605|http://git-wip-us.apache.org/repos/asf/calcite/commit/3c67a605].
> Invalid code generated for negative of byte and short values
>
>
> Key: CALCITE-2180
> URL: https://issues.apache.org/jira/browse/CALCITE-2180
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.16.0
>
>
> Invalid code is generated for negative of byte and short values. The reason
> is that, in Java, if b is a value of type {{byte}}, then {{-b}} has type
> {{int}}; similarly if {{b}} has type {{short}}. The code generator needs to
> accommodate for this.
> The query {code}select -deptno from dept{code} demonstrates the problem,
> since {{deptno}} has SQL type {{TINYINT}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2184) subquery leading to a "java.lang.ClassCastException: RexSubQuery cannot be cast to RexLocalRef"
[
https://issues.apache.org/jira/browse/CALCITE-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alessandro Solimando updated CALCITE-2184:
--
Priority: Major (was: Minor)
> subquery leading to a "java.lang.ClassCastException: RexSubQuery cannot be
> cast to RexLocalRef"
> ---
>
> Key: CALCITE-2184
> URL: https://issues.apache.org/jira/browse/CALCITE-2184
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.16.0
>Reporter: Alessandro Solimando
>Assignee: Julian Hyde
>Priority: Major
>
> Executing this query Calcite tries to cast the subquery (_RexSubQuery_) to a
> local reference (_RexLocalRef_), resulting in a _ClassCastException_.
> Here is the query:
> {code:sql}
> select *
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
> where exists (
> select *
> from (values (1, 'a'), (2, 'b')) as v(w, z)
> where w < x
> )
> {code}
> But the same happens with other (similar) queries, such as the following:
> {code:sql}
> select x
> from (values (1, 'a'), (2, 'b')) as t(x, y)
> where x <= all (
> select x
> from (values (1, 'a'), (2, 'b'), (1, 'b'), (2, 'c'), (2, 'c')) as t(x, y)
> )
> {code}
> Please note that this problem is "masked" by the issues described in
> [CALCITE-2183|https://issues.apache.org/jira/browse/CALCITE-2183], and it can
> be reproduced only by "patching" this second.
> Below the complete stack trace:
> {noformat}
> java.lang.RuntimeException: With materializationsEnabled=false, limit=0
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1346)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1329)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returnsUnordered(CalciteAssert.java:1357)
> at
> org.apache.calcite.test.SparkAdapterTest.commonTester(SparkAdapterTest.java:93)
> at
> org.apache.calcite.test.SparkAdapterTest.testFilterExists(SparkAdapterTest.java:720)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: java.sql.SQLException: Error while executing SQL "select *
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
> where exists (
> select *
> from (values (1, 'a'), (2, 'b')) as v(w, z)
> where w < x
> )": org.apache.calcite.rex.RexSubQuery cannot be cast to
> org.apache.calcite.rex.RexLocalRef
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
> at
>
[jira] [Commented] (CALCITE-508) Reading from ResultSet before calling next() should throw SQLException not NoSuchElementException
[ https://issues.apache.org/jira/browse/CALCITE-508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367922#comment-16367922 ] ASF GitHub Bot commented on CALCITE-508: Github user asolimando commented on the issue: https://github.com/apache/calcite-avatica/pull/23 Hello @vlsi, @julianhyde, can you have a look at the PR? It would be nice to make it before the release of Avatica 1.16.0, so CALCITE-508 could be carried out also on Calcite side. Thanks! > Reading from ResultSet before calling next() should throw SQLException not > NoSuchElementException > - > > Key: CALCITE-508 > URL: https://issues.apache.org/jira/browse/CALCITE-508 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Labels: newbie > > Reading from ResultSet before calling next() should throw SQLException not > NoSuchElementException. > Each of the Cursor.Accessor.getXxx methods should convert runtime exceptions > to SQLException. > JdbcTest.testExtract currently demonstrates this problem; it passes if there > is a NoSuchElementException, but should look for a SQLException. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2185) Additional unit tests for Spark Adapter
Alessandro Solimando created CALCITE-2185: - Summary: Additional unit tests for Spark Adapter Key: CALCITE-2185 URL: https://issues.apache.org/jira/browse/CALCITE-2185 Project: Calcite Issue Type: Test Components: spark Reporter: Alessandro Solimando Assignee: Julian Hyde Add some unit tests covering more aspects of the Spark Adapter. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (CALCITE-2184) subquery leading to a "java.lang.ClassCastException: RexSubQuery cannot be cast to RexLocalRef"
[
https://issues.apache.org/jira/browse/CALCITE-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alessandro Solimando updated CALCITE-2184:
--
Comment: was deleted
(was: The issues is manifest only when the other is fixed)
> subquery leading to a "java.lang.ClassCastException: RexSubQuery cannot be
> cast to RexLocalRef"
> ---
>
> Key: CALCITE-2184
> URL: https://issues.apache.org/jira/browse/CALCITE-2184
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.16.0
>Reporter: Alessandro Solimando
>Assignee: Julian Hyde
>Priority: Minor
>
> Executing this query Calcite tries to cast the subquery (_RexSubQuery_) to a
> local reference (_RexLocalRef_), resulting in a _ClassCastException_.
> Here is the query:
> {code:sql}
> select *
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
> where exists (
> select *
> from (values (1, 'a'), (2, 'b')) as v(w, z)
> where w < x
> )
> {code}
> But the same happens with other (similar) queries, such as the following:
> {code:sql}
> select x
> from (values (1, 'a'), (2, 'b')) as t(x, y)
> where x <= all (
> select x
> from (values (1, 'a'), (2, 'b'), (1, 'b'), (2, 'c'), (2, 'c')) as t(x, y)
> )
> {code}
> Please note that this problem is "masked" by the issues described in
> [CALCITE-2183|https://issues.apache.org/jira/browse/CALCITE-2183], and it can
> be reproduced only by "patching" this second.
> Below the complete stack trace:
> {noformat}
> java.lang.RuntimeException: With materializationsEnabled=false, limit=0
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1346)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1329)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returnsUnordered(CalciteAssert.java:1357)
> at
> org.apache.calcite.test.SparkAdapterTest.commonTester(SparkAdapterTest.java:93)
> at
> org.apache.calcite.test.SparkAdapterTest.testFilterExists(SparkAdapterTest.java:720)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: java.sql.SQLException: Error while executing SQL "select *
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
> where exists (
> select *
> from (values (1, 'a'), (2, 'b')) as v(w, z)
> where w < x
> )": org.apache.calcite.rex.RexSubQuery cannot be cast to
> org.apache.calcite.rex.RexLocalRef
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
> at
>
[jira] [Issue Comment Deleted] (CALCITE-2184) subquery leading to a "java.lang.ClassCastException: RexSubQuery cannot be cast to RexLocalRef"
[
https://issues.apache.org/jira/browse/CALCITE-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alessandro Solimando updated CALCITE-2184:
--
Comment: was deleted
(was: Depends on
[CALCITE-2183|https://issues.apache.org/jira/browse/CALCITE-2183])
> subquery leading to a "java.lang.ClassCastException: RexSubQuery cannot be
> cast to RexLocalRef"
> ---
>
> Key: CALCITE-2184
> URL: https://issues.apache.org/jira/browse/CALCITE-2184
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.16.0
>Reporter: Alessandro Solimando
>Assignee: Julian Hyde
>Priority: Minor
>
> Executing this query Calcite tries to cast the subquery (_RexSubQuery_) to a
> local reference (_RexLocalRef_), resulting in a _ClassCastException_.
> Here is the query:
> {code:sql}
> select *
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
> where exists (
> select *
> from (values (1, 'a'), (2, 'b')) as v(w, z)
> where w < x
> )
> {code}
> But the same happens with other (similar) queries, such as the following:
> {code:sql}
> select x
> from (values (1, 'a'), (2, 'b')) as t(x, y)
> where x <= all (
> select x
> from (values (1, 'a'), (2, 'b'), (1, 'b'), (2, 'c'), (2, 'c')) as t(x, y)
> )
> {code}
> Please note that this problem is "masked" by the issues described in
> [CALCITE-2183|https://issues.apache.org/jira/browse/CALCITE-2183], and it can
> be reproduced only by "patching" this second.
> Below the complete stack trace:
> {noformat}
> java.lang.RuntimeException: With materializationsEnabled=false, limit=0
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1346)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1329)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returnsUnordered(CalciteAssert.java:1357)
> at
> org.apache.calcite.test.SparkAdapterTest.commonTester(SparkAdapterTest.java:93)
> at
> org.apache.calcite.test.SparkAdapterTest.testFilterExists(SparkAdapterTest.java:720)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: java.sql.SQLException: Error while executing SQL "select *
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
> where exists (
> select *
> from (values (1, 'a'), (2, 'b')) as v(w, z)
> where w < x
> )": org.apache.calcite.rex.RexSubQuery cannot be cast to
> org.apache.calcite.rex.RexLocalRef
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> at
>
[jira] [Commented] (CALCITE-2183) invocation of copy method in RelSubset class
[
https://issues.apache.org/jira/browse/CALCITE-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367793#comment-16367793
]
Alessandro Solimando commented on CALCITE-2183:
---
Discussion on the dev list
[here|http://mail-archives.apache.org/mod_mbox/calcite-dev/201802.mbox/%3CCAFtOckKqvJBRfR9_6GLYtPZa3QZVWK9YfTQ3ZFFZtHn_K0%2BSxA%40mail.gmail.com%3E]
> invocation of copy method in RelSubset class
>
>
> Key: CALCITE-2183
> URL: https://issues.apache.org/jira/browse/CALCITE-2183
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.16.0
>Reporter: Alessandro Solimando
>Assignee: Julian Hyde
>Priority: Minor
>
> Consider the following query:
> {code:sql}
> select *
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
> where x between 3 and 4
> {code}
> When executed, an exception is thrown (the full stack trace is at the end) in
> _copy_ method in _RelSubset_ class, as the method is not meant to be called.
>
> This happens (in this particular case) while Calcite is trying to get rid of
> unused terms (specifically, _trimUnusedFields_ method from
> _SqlToRelConverted_ class).
> This is problematic as the trace of calls is legitimate, so the method should
> be executable.
> Complete stack trace documenting the issue:
> {noformat}
> java.lang.RuntimeException: With materializationsEnabled=false, limit=0
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1346)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1329)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returnsUnordered(CalciteAssert.java:1357)
> at
> org.apache.calcite.test.SparkAdapterTest.commonTester(SparkAdapterTest.java:93)
> at
> org.apache.calcite.test.SparkAdapterTest.testFilterBetween(SparkAdapterTest.java:460)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: java.sql.SQLException: Error while executing SQL "select *
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
> where x between 3 and 4": null
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:568)
> ... 27 more
> Caused by: java.lang.UnsupportedOperationException
> at org.apache.calcite.plan.volcano.RelSubset.copy(RelSubset.java:149)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.trimUnusedFields(SqlToRelConverter.java:517)
> at
[jira] [Updated] (CALCITE-2184) subquery leading to a "java.lang.ClassCastException: RexSubQuery cannot be cast to RexLocalRef"
[
https://issues.apache.org/jira/browse/CALCITE-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alessandro Solimando updated CALCITE-2184:
--
External issue URL: (was:
https://issues.apache.org/jira/browse/CALCITE-2183)
External issue ID: (was: CALCITE-2183)
Depends on [CALCITE-2183|https://issues.apache.org/jira/browse/CALCITE-2183]
> subquery leading to a "java.lang.ClassCastException: RexSubQuery cannot be
> cast to RexLocalRef"
> ---
>
> Key: CALCITE-2184
> URL: https://issues.apache.org/jira/browse/CALCITE-2184
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.16.0
>Reporter: Alessandro Solimando
>Assignee: Julian Hyde
>Priority: Minor
>
> Executing this query Calcite tries to cast the subquery (_RexSubQuery_) to a
> local reference (_RexLocalRef_), resulting in a _ClassCastException_.
> Here is the query:
> {code:sql}
> select *
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
> where exists (
> select *
> from (values (1, 'a'), (2, 'b')) as v(w, z)
> where w < x
> )
> {code}
> But the same happens with other (similar) queries, such as the following:
> {code:sql}
> select x
> from (values (1, 'a'), (2, 'b')) as t(x, y)
> where x <= all (
> select x
> from (values (1, 'a'), (2, 'b'), (1, 'b'), (2, 'c'), (2, 'c')) as t(x, y)
> )
> {code}
> Please note that this problem is "masked" by the issues described in
> [CALCITE-2183|https://issues.apache.org/jira/browse/CALCITE-2183], and it can
> be reproduced only by "patching" this second.
> Below the complete stack trace:
> {noformat}
> java.lang.RuntimeException: With materializationsEnabled=false, limit=0
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1346)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1329)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returnsUnordered(CalciteAssert.java:1357)
> at
> org.apache.calcite.test.SparkAdapterTest.commonTester(SparkAdapterTest.java:93)
> at
> org.apache.calcite.test.SparkAdapterTest.testFilterExists(SparkAdapterTest.java:720)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: java.sql.SQLException: Error while executing SQL "select *
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
> where exists (
> select *
> from (values (1, 'a'), (2, 'b')) as v(w, z)
> where w < x
> )": org.apache.calcite.rex.RexSubQuery cannot be cast to
> org.apache.calcite.rex.RexLocalRef
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
>
[jira] [Updated] (CALCITE-2184) subquery leading to a "java.lang.ClassCastException: RexSubQuery cannot be cast to RexLocalRef"
[
https://issues.apache.org/jira/browse/CALCITE-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alessandro Solimando updated CALCITE-2184:
--
External issue URL: https://issues.apache.org/jira/browse/CALCITE-2183
External issue ID: CALCITE-2183
> subquery leading to a "java.lang.ClassCastException: RexSubQuery cannot be
> cast to RexLocalRef"
> ---
>
> Key: CALCITE-2184
> URL: https://issues.apache.org/jira/browse/CALCITE-2184
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.16.0
>Reporter: Alessandro Solimando
>Assignee: Julian Hyde
>Priority: Minor
>
> Executing this query Calcite tries to cast the subquery (_RexSubQuery_) to a
> local reference (_RexLocalRef_), resulting in a _ClassCastException_.
> Here is the query:
> {code:sql}
> select *
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
> where exists (
> select *
> from (values (1, 'a'), (2, 'b')) as v(w, z)
> where w < x
> )
> {code}
> But the same happens with other (similar) queries, such as the following:
> {code:sql}
> select x
> from (values (1, 'a'), (2, 'b')) as t(x, y)
> where x <= all (
> select x
> from (values (1, 'a'), (2, 'b'), (1, 'b'), (2, 'c'), (2, 'c')) as t(x, y)
> )
> {code}
> Please note that this problem is "masked" by the issues described in
> [CALCITE-2183|https://issues.apache.org/jira/browse/CALCITE-2183], and it can
> be reproduced only by "patching" this second.
> Below the complete stack trace:
> {noformat}
> java.lang.RuntimeException: With materializationsEnabled=false, limit=0
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1346)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1329)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returnsUnordered(CalciteAssert.java:1357)
> at
> org.apache.calcite.test.SparkAdapterTest.commonTester(SparkAdapterTest.java:93)
> at
> org.apache.calcite.test.SparkAdapterTest.testFilterExists(SparkAdapterTest.java:720)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: java.sql.SQLException: Error while executing SQL "select *
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
> where exists (
> select *
> from (values (1, 'a'), (2, 'b')) as v(w, z)
> where w < x
> )": org.apache.calcite.rex.RexSubQuery cannot be cast to
> org.apache.calcite.rex.RexLocalRef
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> at
>
[jira] [Created] (CALCITE-2184) subquery leading to a "java.lang.ClassCastException: RexSubQuery cannot be cast to RexLocalRef"
Alessandro Solimando created CALCITE-2184:
-
Summary: subquery leading to a "java.lang.ClassCastException:
RexSubQuery cannot be cast to RexLocalRef"
Key: CALCITE-2184
URL: https://issues.apache.org/jira/browse/CALCITE-2184
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.16.0
Reporter: Alessandro Solimando
Assignee: Julian Hyde
Executing this query Calcite tries to cast the subquery (_RexSubQuery_) to a
local reference (_RexLocalRef_), resulting in a _ClassCastException_.
Here is the query:
{code:sql}
select *
from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
where exists (
select *
from (values (1, 'a'), (2, 'b')) as v(w, z)
where w < x
)
{code}
But the same happens with other (similar) queries, such as the following:
{code:sql}
select x
from (values (1, 'a'), (2, 'b')) as t(x, y)
where x <= all (
select x
from (values (1, 'a'), (2, 'b'), (1, 'b'), (2, 'c'), (2, 'c')) as t(x, y)
)
{code}
Please note that this problem is "masked" by the issues described in
[CALCITE-2183|https://issues.apache.org/jira/browse/CALCITE-2183], and it can
be reproduced only by "patching" this second.
Below the complete stack trace:
{noformat}
java.lang.RuntimeException: With materializationsEnabled=false, limit=0
at
org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1346)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1329)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returnsUnordered(CalciteAssert.java:1357)
at
org.apache.calcite.test.SparkAdapterTest.commonTester(SparkAdapterTest.java:93)
at
org.apache.calcite.test.SparkAdapterTest.testFilterExists(SparkAdapterTest.java:720)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.sql.SQLException: Error while executing SQL "select *
from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
where exists (
select *
from (values (1, 'a'), (2, 'b')) as v(w, z)
where w < x
)": org.apache.calcite.rex.RexSubQuery cannot be cast to
org.apache.calcite.rex.RexLocalRef
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
at
org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:568)
... 27 more
Caused by: java.lang.ClassCastException: org.apache.calcite.rex.RexSubQuery
cannot be cast to org.apache.calcite.rex.RexLocalRef
at
org.apache.calcite.rex.RexProgramBuilder.registerInput(RexProgramBuilder.java:298)
at
org.apache.calcite.rex.RexProgramBuilder.addCondition(RexProgramBuilder.java:272)
at
[jira] [Created] (CALCITE-2183) invocation of copy method in RelSubset class
Alessandro Solimando created CALCITE-2183:
-
Summary: invocation of copy method in RelSubset class
Key: CALCITE-2183
URL: https://issues.apache.org/jira/browse/CALCITE-2183
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.16.0
Reporter: Alessandro Solimando
Assignee: Julian Hyde
Consider the following query:
{code:sql}
select *
from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
where x between 3 and 4
{code}
When executed, an exception is thrown (the full stack trace is at the end) in
_copy_ method in _RelSubset_ class, as the method is not meant to be called.
This happens (in this particular case) while Calcite is trying to get rid of
unused terms (specifically, _trimUnusedFields_ method from _SqlToRelConverted_
class).
This is problematic as the trace of calls is legitimate, so the method should
be executable.
Complete stack trace documenting the issue:
{noformat}
java.lang.RuntimeException: With materializationsEnabled=false, limit=0
at
org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1346)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1329)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returnsUnordered(CalciteAssert.java:1357)
at
org.apache.calcite.test.SparkAdapterTest.commonTester(SparkAdapterTest.java:93)
at
org.apache.calcite.test.SparkAdapterTest.testFilterBetween(SparkAdapterTest.java:460)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.sql.SQLException: Error while executing SQL "select *
from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
where x between 3 and 4": null
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
at
org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:568)
... 27 more
Caused by: java.lang.UnsupportedOperationException
at org.apache.calcite.plan.volcano.RelSubset.copy(RelSubset.java:149)
at
org.apache.calcite.sql2rel.SqlToRelConverter.trimUnusedFields(SqlToRelConverter.java:517)
at org.apache.calcite.prepare.Prepare.trimUnusedFields(Prepare.java:391)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:304)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:230)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:781)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610)
at
[jira] [Commented] (CALCITE-2170) Use Druid Expressions capabilities to improve the amount of work that can be pushed to Druid
[
https://issues.apache.org/jira/browse/CALCITE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367738#comment-16367738
]
slim bouguerra commented on CALCITE-2170:
-
[~julianhyde] I am sorry to hear that this is causing frustration. I agree with
you, contributions should not be framed as `take or leave it and it` is not my
intent anyway.
> Use Druid Expressions capabilities to improve the amount of work that can be
> pushed to Druid
>
>
> Key: CALCITE-2170
> URL: https://issues.apache.org/jira/browse/CALCITE-2170
> Project: Calcite
> Issue Type: New Feature
> Components: druid
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
>
> Druid 0.11 has newly built in capabilities called Expressions that can be
> used to push expression like projects/aggregates/filters.
> In order to leverage this new feature, some changes need to be done to the
> Druid Calcite adapter.
> This is a link to the current supported functions and expressions in Druid
> [http://druid.io/docs/latest/misc/math-expr.html]
> As you can see from the Docs an expression can be an actual tree of operators,
> Expression can be used with Filters, Projects, Aggregates, PostAggregates and
> Having filters. For Filters will have new Filter kind called Filter
> expression.
> FYI, you might ask can we push everything as Expression Filter the short
> answer
> is no because, other kinds of Druid filters perform better when used, Hence
> Expression filter is a plan B sort of thing. In order to push expression as
> Projects and Aggregates we will be using Expression based Virtual Columns.
> The major change is the merging of the logic of pushdown verification code and
> the Translation of RexCall/RexNode to Druid Json, native physical language.
> The
> main drive behind this redesign is the fact that in order to check if we can
> push down a tree of expressions to Druid we have to compute the Druid
> Expression
> String anyway. Thus instead of having 2 different code paths, one for pushdown
> validation and one for Json generation we can have one function that does
> both.
> For instance instead of having one code path to test and check if a given
> filter
> can be pushed or not and then having a translation layer code, will have
> one function that either returns a valid Druid Filter or null if it is not
> possible to pushdown. The same idea will be applied to how we push Projects
> and
> Aggregates, Post Aggregates and Sort.
> Here are the main elements/Classes of the new design. First will be merging
> the logic of
> Translation of Literals/InputRex/RexCall to a Druid physical representation.
> Translate leaf RexNode to Valid pair Druid Column + Extraction functions if
> possible
> {code:java}
> /**
> * @param rexNode leaf Input Ref to Druid Column
> * @param rowType row type
> * @param druidQuery druid query
> *
> * @return {@link Pair} of Column name and Extraction Function on the top of
> the input ref or
> * {@link Pair of(null, null)} when can not translate to valid Druid column
> */
> protected static Pair toDruidColumn(RexNode
> rexNode,
> RelDataType rowType, DruidQuery druidQuery
> )
> {code}
> In the other hand, in order to Convert Literals to Druid Literals will
> introduce
> {code:java}
> /**
> * @param rexNode rexNode to translate to Druid literal equivalante
> * @param rowType rowType associated to rexNode
> * @param druidQuery druid Query
> *
> * @return non null string or null if it can not translate to valid Druid
> equivalent
> */
> @Nullable
> private static String toDruidLiteral(RexNode rexNode, RelDataType rowType,
> DruidQuery druidQuery
> )
> {code}
> Main new functions used to pushdown nodes and Druid Json generation.
> Filter pushdown verification and generates is done via
> {code:java}
> org.apache.calcite.adapter.druid.DruidJsonFilter#toDruidFilters
> {code}
> For project pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeProjectAsScan.
> {code}
> For Grouping pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeProjectGroupSet.
> {code}
> For Aggregation pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeDruidJsonAgg
> {code}
> For sort pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeSort\{code}
> Pushing of PostAggregates will be using Expression post Aggregates and use
> {code}
> org.apache.calcite.adapter.druid.DruidExpressions#toDruidExpression\{code}
> to generate expression
> For Expression computation most of the work is done here
> {code:java}
> org.apache.calcite.adapter.druid.DruidExpressions#toDruidExpression\{code}
> This static function
[jira] [Commented] (CALCITE-2170) Use Druid Expressions capabilities to improve the amount of work that can be pushed to Druid
[
https://issues.apache.org/jira/browse/CALCITE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367733#comment-16367733
]
Jesus Camacho Rodriguez commented on CALCITE-2170:
--
[~julianhyde], nevertheless I think the contribution is useful/valuable, that
is why I wanted to merge it and asked for your feedback about the PR. Approach
could have been more general and reusable across different adapters, but since
there is a PR already in place, I think we should log a follow-up JIRA for that
and continue the discussion there. Current tests would give us a baseline to
what should be supported, then some work would have to be done to refactor
existing code in Druid adapter.
All changes in the patch concentrate in Druid adapter, I would probably
advocate for a different solution if we were making changes to Calcite core
APIs.
> Use Druid Expressions capabilities to improve the amount of work that can be
> pushed to Druid
>
>
> Key: CALCITE-2170
> URL: https://issues.apache.org/jira/browse/CALCITE-2170
> Project: Calcite
> Issue Type: New Feature
> Components: druid
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
>
> Druid 0.11 has newly built in capabilities called Expressions that can be
> used to push expression like projects/aggregates/filters.
> In order to leverage this new feature, some changes need to be done to the
> Druid Calcite adapter.
> This is a link to the current supported functions and expressions in Druid
> [http://druid.io/docs/latest/misc/math-expr.html]
> As you can see from the Docs an expression can be an actual tree of operators,
> Expression can be used with Filters, Projects, Aggregates, PostAggregates and
> Having filters. For Filters will have new Filter kind called Filter
> expression.
> FYI, you might ask can we push everything as Expression Filter the short
> answer
> is no because, other kinds of Druid filters perform better when used, Hence
> Expression filter is a plan B sort of thing. In order to push expression as
> Projects and Aggregates we will be using Expression based Virtual Columns.
> The major change is the merging of the logic of pushdown verification code and
> the Translation of RexCall/RexNode to Druid Json, native physical language.
> The
> main drive behind this redesign is the fact that in order to check if we can
> push down a tree of expressions to Druid we have to compute the Druid
> Expression
> String anyway. Thus instead of having 2 different code paths, one for pushdown
> validation and one for Json generation we can have one function that does
> both.
> For instance instead of having one code path to test and check if a given
> filter
> can be pushed or not and then having a translation layer code, will have
> one function that either returns a valid Druid Filter or null if it is not
> possible to pushdown. The same idea will be applied to how we push Projects
> and
> Aggregates, Post Aggregates and Sort.
> Here are the main elements/Classes of the new design. First will be merging
> the logic of
> Translation of Literals/InputRex/RexCall to a Druid physical representation.
> Translate leaf RexNode to Valid pair Druid Column + Extraction functions if
> possible
> {code:java}
> /**
> * @param rexNode leaf Input Ref to Druid Column
> * @param rowType row type
> * @param druidQuery druid query
> *
> * @return {@link Pair} of Column name and Extraction Function on the top of
> the input ref or
> * {@link Pair of(null, null)} when can not translate to valid Druid column
> */
> protected static Pair toDruidColumn(RexNode
> rexNode,
> RelDataType rowType, DruidQuery druidQuery
> )
> {code}
> In the other hand, in order to Convert Literals to Druid Literals will
> introduce
> {code:java}
> /**
> * @param rexNode rexNode to translate to Druid literal equivalante
> * @param rowType rowType associated to rexNode
> * @param druidQuery druid Query
> *
> * @return non null string or null if it can not translate to valid Druid
> equivalent
> */
> @Nullable
> private static String toDruidLiteral(RexNode rexNode, RelDataType rowType,
> DruidQuery druidQuery
> )
> {code}
> Main new functions used to pushdown nodes and Druid Json generation.
> Filter pushdown verification and generates is done via
> {code:java}
> org.apache.calcite.adapter.druid.DruidJsonFilter#toDruidFilters
> {code}
> For project pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeProjectAsScan.
> {code}
> For Grouping pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeProjectGroupSet.
> {code}
> For Aggregation pushdown added
> {code:java}
>
[jira] [Commented] (CALCITE-2182) Release Avatica 1.11
[ https://issues.apache.org/jira/browse/CALCITE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367714#comment-16367714 ] Julian Hyde commented on CALCITE-2182: -- I think it would be good to release Avatica 1.11, then make Calcite 1.16 depend upon it. Both need to update versions of dependencies, e.g. Jackson. > Release Avatica 1.11 > > > Key: CALCITE-2182 > URL: https://issues.apache.org/jira/browse/CALCITE-2182 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > > Release Avatica 1.11. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2182) Release Avatica 1.11
Julian Hyde created CALCITE-2182: Summary: Release Avatica 1.11 Key: CALCITE-2182 URL: https://issues.apache.org/jira/browse/CALCITE-2182 Project: Calcite Issue Type: Bug Reporter: Julian Hyde Assignee: Julian Hyde Release Avatica 1.11. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2170) Use Druid Expressions capabilities to improve the amount of work that can be pushed to Druid
[
https://issues.apache.org/jira/browse/CALCITE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367702#comment-16367702
]
Julian Hyde commented on CALCITE-2170:
--
This is frustrating. There are clear signs of off-list development here. A JIRA
case logged, and a pull request very shortly thereafter. I responded quickly to
the JIRA case suggesting a way to approach this, but Slim's PR took a different
tack. I have looked at the PR, raised concerns, and they haven't been
addressed. I understand that Slim isn't particularly inclined to re-implement
everything he's done. I'd like there to be some give and take, but since you've
done off-list development it's "take it or leave it".
> Use Druid Expressions capabilities to improve the amount of work that can be
> pushed to Druid
>
>
> Key: CALCITE-2170
> URL: https://issues.apache.org/jira/browse/CALCITE-2170
> Project: Calcite
> Issue Type: New Feature
> Components: druid
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
>
> Druid 0.11 has newly built in capabilities called Expressions that can be
> used to push expression like projects/aggregates/filters.
> In order to leverage this new feature, some changes need to be done to the
> Druid Calcite adapter.
> This is a link to the current supported functions and expressions in Druid
> [http://druid.io/docs/latest/misc/math-expr.html]
> As you can see from the Docs an expression can be an actual tree of operators,
> Expression can be used with Filters, Projects, Aggregates, PostAggregates and
> Having filters. For Filters will have new Filter kind called Filter
> expression.
> FYI, you might ask can we push everything as Expression Filter the short
> answer
> is no because, other kinds of Druid filters perform better when used, Hence
> Expression filter is a plan B sort of thing. In order to push expression as
> Projects and Aggregates we will be using Expression based Virtual Columns.
> The major change is the merging of the logic of pushdown verification code and
> the Translation of RexCall/RexNode to Druid Json, native physical language.
> The
> main drive behind this redesign is the fact that in order to check if we can
> push down a tree of expressions to Druid we have to compute the Druid
> Expression
> String anyway. Thus instead of having 2 different code paths, one for pushdown
> validation and one for Json generation we can have one function that does
> both.
> For instance instead of having one code path to test and check if a given
> filter
> can be pushed or not and then having a translation layer code, will have
> one function that either returns a valid Druid Filter or null if it is not
> possible to pushdown. The same idea will be applied to how we push Projects
> and
> Aggregates, Post Aggregates and Sort.
> Here are the main elements/Classes of the new design. First will be merging
> the logic of
> Translation of Literals/InputRex/RexCall to a Druid physical representation.
> Translate leaf RexNode to Valid pair Druid Column + Extraction functions if
> possible
> {code:java}
> /**
> * @param rexNode leaf Input Ref to Druid Column
> * @param rowType row type
> * @param druidQuery druid query
> *
> * @return {@link Pair} of Column name and Extraction Function on the top of
> the input ref or
> * {@link Pair of(null, null)} when can not translate to valid Druid column
> */
> protected static Pair toDruidColumn(RexNode
> rexNode,
> RelDataType rowType, DruidQuery druidQuery
> )
> {code}
> In the other hand, in order to Convert Literals to Druid Literals will
> introduce
> {code:java}
> /**
> * @param rexNode rexNode to translate to Druid literal equivalante
> * @param rowType rowType associated to rexNode
> * @param druidQuery druid Query
> *
> * @return non null string or null if it can not translate to valid Druid
> equivalent
> */
> @Nullable
> private static String toDruidLiteral(RexNode rexNode, RelDataType rowType,
> DruidQuery druidQuery
> )
> {code}
> Main new functions used to pushdown nodes and Druid Json generation.
> Filter pushdown verification and generates is done via
> {code:java}
> org.apache.calcite.adapter.druid.DruidJsonFilter#toDruidFilters
> {code}
> For project pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeProjectAsScan.
> {code}
> For Grouping pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeProjectGroupSet.
> {code}
> For Aggregation pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeDruidJsonAgg
> {code}
> For sort pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeSort\{code}
> Pushing of
[jira] [Created] (CALCITE-2181) Release Calcite 1.16.0
Jesus Camacho Rodriguez created CALCITE-2181: Summary: Release Calcite 1.16.0 Key: CALCITE-2181 URL: https://issues.apache.org/jira/browse/CALCITE-2181 Project: Calcite Issue Type: Task Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Release Calcite 1.16.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
