[jira] [Commented] (CALCITE-2045) Support "CREATE TYPE" DDL
[
https://issues.apache.org/jira/browse/CALCITE-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356593#comment-16356593
]
Julian Hyde commented on CALCITE-2045:
--
Don't add that kind of stuff to {{SqlDataTypeSpec}}. As a sub-class of
{{SqlNode}}, it is an AST node, and should only have the information in the raw
SQL; nothing should be 'resolved'.
It is the validator's job to figure out types. For an expression, you call
{{RelDataType SqlValidator.getValidatedNodeType(SqlNode)}}. For a
{{SqlDataTypeSpec}}, you could actually call the same method. It will use the
{{SqlValidatorCatalogReader}} inside the validator. This has methods
{{CalciteSchema getRootSchema()}} and {{RelDataType getNamedType(SqlIdentifier
typeName)}}, both of which you will find useful.
> Support "CREATE TYPE" DDL
> -
>
> Key: CALCITE-2045
> URL: https://issues.apache.org/jira/browse/CALCITE-2045
> Project: Calcite
> Issue Type: New Feature
> Components: core
>Reporter: Shuyi Chen
>Assignee: Shuyi Chen
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2053) Overloaded user-defined functions that have Double and BigDecimal arguments will goes wrong
[
https://issues.apache.org/jira/browse/CALCITE-2053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356590#comment-16356590
]
yuqi commented on CALCITE-2053:
---
Thanks for your time.
> Overloaded user-defined functions that have Double and BigDecimal arguments
> will goes wrong
> ---
>
> Key: CALCITE-2053
> URL: https://issues.apache.org/jira/browse/CALCITE-2053
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.15.0
>Reporter: yuqi
>Assignee: Julian Hyde
>Priority: Major
>
> We add the following udf method in Smalls.AllTypesFunction
> {code:java}
>public double toDouble(BigDecimal var) {
> return var.doubleValue();
>}
>public double toDouble(Double var) {
> return var;
> }
>
> {code}
> when test it :
> {code:java}
> @Test
> public void testBigDecimalAndLong() {
> final CalciteAssert.AssertThat with = withUdf();
> with.query("values \"adhoc\".\"toDouble\"(cast(1.0 as double))")
> .returns("EXPR$0=1.0\n");
> }{code}
> where price is a double value in table tb, exception occurs:
> {code:java}
> java.lang.AssertionError: DECIMAL(19, 0) at
> org.apache.calcite.sql.type.SqlTypeExplicitPrecedenceList.compareTypePrecedence(SqlTypeExplicitPrecedenceList.java:154)
> at org.apache.calcite.sql.SqlUtil.bestMatch(SqlUtil.java:626) at
> org.apache.calcite.sql.SqlUtil.filterRoutinesByTypePrecedence(SqlUtil.java:592)
> at org.apache.calcite.sql.SqlUtil.lookupSubjectRoutines(SqlUtil.java:446) at
> org.apache.calcite.sql.SqlUtil.lookupRoutine(SqlUtil.java:371) at
> org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:245) at
> org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:223) at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5432)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5419)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1606)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1591)
> at org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:236) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:407) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5136)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:115) at
> org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:235) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:407) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5136)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:115) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:903)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:613)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:553)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:264) at
> org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:230) at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:781)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:221)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
> at org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:568)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1346)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1329)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1293)
> at org.apache.calcite.test.UdfTest.testBigDecimalAndLong(UdfTest.java:891)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497) at
>
[jira] [Comment Edited] (CALCITE-2045) Support "CREATE TYPE" DDL
[ https://issues.apache.org/jira/browse/CALCITE-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356429#comment-16356429 ] Shuyi Chen edited comment on CALCITE-2045 at 2/8/18 3:15 AM: - Hi [~julianhyde], I think I'll need to extend SqlDataTypeSpec to pass the schema (or something equivalence to provide access to the registered UDT) for SqlDataTypeSpec.deriveType(). Otherwise, after I create a type using "_CREATE TYPE mytype1 as int_", when I use the type to, say, _create table t (c mytype1),_ it will fail. Because the type info is only registered in the schema, and SqlDataTypeSpec has to access the created user-defined type through schema. However, this might be difficult because not all call sites of SqlDataTypeSpec.deriveType can provide access to the schema. If this is the right option, I can try to modify all call sites to provide access to the schema directly or indirectly. Do you have any suggestions? was (Author: suez1224): Hi [~julianhyde], I think I'll need to extend SqlDataTypeSpec to pass the schema (or something equivalence to provide access to the registered UDT) during SqlDataTypeSpec.deriveType call. Otherwise, after I create a type using "_CREATE TYPE mytype1 as int_", when I use the type to, say, _create table t (c mytype1),_ it will fail. Because the type info is only registered in the schema, and SqlDataTypeSpec has to access the created user-defined type through schema. However, this might be difficult because not all call sites of SqlDataTypeSpec.deriveType can provide access to the schema. If this is the right option, I can try to modify all call sites to provide access to the schema directly or indirectly. Do you have any suggestions? > Support "CREATE TYPE" DDL > - > > Key: CALCITE-2045 > URL: https://issues.apache.org/jira/browse/CALCITE-2045 > Project: Calcite > Issue Type: New Feature > Components: core >Reporter: Shuyi Chen >Assignee: Shuyi Chen >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2045) Support "CREATE TYPE" DDL
[ https://issues.apache.org/jira/browse/CALCITE-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356429#comment-16356429 ] Shuyi Chen commented on CALCITE-2045: - Hi [~julianhyde], I think I'll need to extend SqlDataTypeSpec to pass the schema (or something equivalence to provide access to the registered UDT) during SqlDataTypeSpec.deriveType call. Otherwise, after I create a type using "_CREATE TYPE mytype1 as int_", when I use the type to, say, _create table t (c mytype1),_ it will fail. Because the type info is only registered in the schema, and SqlDataTypeSpec has to access the created user-defined type through schema. However, this might be difficult because not all call sites of SqlDataTypeSpec.deriveType can provide access to the schema. If this is the right option, I can try to modify all call sites to provide access to the schema directly or indirectly. Do you have any suggestions? > Support "CREATE TYPE" DDL > - > > Key: CALCITE-2045 > URL: https://issues.apache.org/jira/browse/CALCITE-2045 > Project: Calcite > Issue Type: New Feature > Components: core >Reporter: Shuyi Chen >Assignee: Shuyi Chen >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2172) Document IDE setup for project contribution
Edmon Begoli created CALCITE-2172: - Summary: Document IDE setup for project contribution Key: CALCITE-2172 URL: https://issues.apache.org/jira/browse/CALCITE-2172 Project: Calcite Issue Type: Improvement Components: site Reporter: Edmon Begoli Assignee: Edmon Begoli Document the IDE setup for Calcite development. Start with IDEA, document Eclipse, NetBeans, and maybe VIM and Emacs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2059) Apache Geode adapter
[ https://issues.apache.org/jira/browse/CALCITE-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356251#comment-16356251 ] Michael Mior commented on CALCITE-2059: --- [~tzolov] If you're referring to the JDK 9 build on Travis, that's a known issue and not a concern now as long the tests pass locally. > Apache Geode adapter > > > Key: CALCITE-2059 > URL: https://issues.apache.org/jira/browse/CALCITE-2059 > Project: Calcite > Issue Type: New Feature > Components: geode >Reporter: Christian Tzolov >Assignee: Julian Hyde >Priority: Major > > I've been working on a Calcite adapter for [Apache > Geode|http://geode.apache.org]. > Current implementation uses the plain Geode API and > [OQL|http://geode.apache.org/docs/guide/13/developing/querying_basics/chapter_overview.html](Object > Query Interface) to push down relational expressions such as projections, > filtering, sorting, and grouping . > Provided functionality can hopefully address certain Geode use cases and will > provide a stepping stone for future improvements. > Here are some remaining tasks as i see it: > * New tests for test suite (and update calcite-test-dataset to support Geode) > * Add Integration tests that use calcite-test-dataset > * Documentation -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2170) Use Druid Expressions capabilities to improve the amount of work that can be pushed to Druid
[
https://issues.apache.org/jira/browse/CALCITE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356201#comment-16356201
]
Julian Hyde commented on CALCITE-2170:
--
Well, there are two algebras: {{RelNode}} for relational expressions and a
sub-algebra {{RexNode}} for scalar expressions. We both agree that we should
stay within the {{RelNode}} algebra. I am arguing that we should also stay in
{{RexNode}} algebra. We've gone the other route before (e.g. in the Splunk
adapter) and the approach runs out of road because you don't have tools such as
{{RexVisitor}} and {{RelOptUtil.splitFilters(RexNode)}} to work on the
expressions.
> Use Druid Expressions capabilities to improve the amount of work that can be
> pushed to Druid
>
>
> Key: CALCITE-2170
> URL: https://issues.apache.org/jira/browse/CALCITE-2170
> Project: Calcite
> Issue Type: New Feature
> Components: druid
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
>
> Druid 0.11 has newly built in capabilities called Expressions that can be
> used to push expression like projects/aggregates/filters.
> In order to leverage this new feature, some changes need to be done to the
> Druid Calcite adapter.
> This is a link to the current supported functions and expressions in Druid
> [http://druid.io/docs/latest/misc/math-expr.html]
> As you can see from the Docs an expression can be an actual tree of operators,
> Expression can be used with Filters, Projects, Aggregates, PostAggregates and
> Having filters. For Filters will have new Filter kind called Filter
> expression.
> FYI, you might ask can we push everything as Expression Filter the short
> answer
> is no because, other kinds of Druid filters perform better when used, Hence
> Expression filter is a plan B sort of thing. In order to push expression as
> Projects and Aggregates we will be using Expression based Virtual Columns.
> The major change is the merging of the logic of pushdown verification code and
> the Translation of RexCall/RexNode to Druid Json, native physical language.
> The
> main drive behind this redesign is the fact that in order to check if we can
> push down a tree of expressions to Druid we have to compute the Druid
> Expression
> String anyway. Thus instead of having 2 different code paths, one for pushdown
> validation and one for Json generation we can have one function that does
> both.
> For instance instead of having one code path to test and check if a given
> filter
> can be pushed or not and then having a translation layer code, will have
> one function that either returns a valid Druid Filter or null if it is not
> possible to pushdown. The same idea will be applied to how we push Projects
> and
> Aggregates, Post Aggregates and Sort.
> Here are the main elements/Classes of the new design. First will be merging
> the logic of
> Translation of Literals/InputRex/RexCall to a Druid physical representation.
> Translate leaf RexNode to Valid pair Druid Column + Extraction functions if
> possible
> {code:java}
> /**
> * @param rexNode leaf Input Ref to Druid Column
> * @param rowType row type
> * @param druidQuery druid query
> *
> * @return {@link Pair} of Column name and Extraction Function on the top of
> the input ref or
> * {@link Pair of(null, null)} when can not translate to valid Druid column
> */
> protected static Pair toDruidColumn(RexNode
> rexNode,
> RelDataType rowType, DruidQuery druidQuery
> )
> {code}
> In the other hand, in order to Convert Literals to Druid Literals will
> introduce
> {code:java}
> /**
> * @param rexNode rexNode to translate to Druid literal equivalante
> * @param rowType rowType associated to rexNode
> * @param druidQuery druid Query
> *
> * @return non null string or null if it can not translate to valid Druid
> equivalent
> */
> @Nullable
> private static String toDruidLiteral(RexNode rexNode, RelDataType rowType,
> DruidQuery druidQuery
> )
> {code}
> Main new functions used to pushdown nodes and Druid Json generation.
> Filter pushdown verification and generates is done via
> {code:java}
> org.apache.calcite.adapter.druid.DruidJsonFilter#toDruidFilters
> {code}
> For project pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeProjectAsScan.
> {code}
> For Grouping pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeProjectGroupSet.
> {code}
> For Aggregation pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeDruidJsonAgg
> {code}
> For sort pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeSort\{code}
> Pushing of PostAggregates will be using Expression post Aggregates
[jira] [Commented] (CALCITE-2059) Apache Geode adapter
[ https://issues.apache.org/jira/browse/CALCITE-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356053#comment-16356053 ] Julian Hyde commented on CALCITE-2059: -- I've made some cosmetic changes; can you review and squash in https://github.com/julianhyde/calcite/tree/2059-geode. Review comments: * Can you explain (in the javadoc preferably) why "GeodeAggregationSort" contains the word "Aggregation"? I imagine it's a Geode concept, but it is misleadingly similar to Calcite's "Aggregate" concept. * In GeodeUtils, the fields REGION_MAP and JAVA_TYPE_FACTORY are a concern because they may cause a memory leak. * If you intend to keep Main1Foo, can you give it a better name. > Apache Geode adapter > > > Key: CALCITE-2059 > URL: https://issues.apache.org/jira/browse/CALCITE-2059 > Project: Calcite > Issue Type: New Feature > Components: geode >Reporter: Christian Tzolov >Assignee: Julian Hyde >Priority: Major > > I've been working on a Calcite adapter for [Apache > Geode|http://geode.apache.org]. > Current implementation uses the plain Geode API and > [OQL|http://geode.apache.org/docs/guide/13/developing/querying_basics/chapter_overview.html](Object > Query Interface) to push down relational expressions such as projections, > filtering, sorting, and grouping . > Provided functionality can hopefully address certain Geode use cases and will > provide a stepping stone for future improvements. > Here are some remaining tasks as i see it: > * New tests for test suite (and update calcite-test-dataset to support Geode) > * Add Integration tests that use calcite-test-dataset > * Documentation -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2059) Apache Geode adapter
[ https://issues.apache.org/jira/browse/CALCITE-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde updated CALCITE-2059: - Component/s: geode > Apache Geode adapter > > > Key: CALCITE-2059 > URL: https://issues.apache.org/jira/browse/CALCITE-2059 > Project: Calcite > Issue Type: New Feature > Components: geode >Reporter: Christian Tzolov >Assignee: Julian Hyde >Priority: Major > > I've been working on a Calcite adapter for [Apache > Geode|http://geode.apache.org]. > Current implementation uses the plain Geode API and > [OQL|http://geode.apache.org/docs/guide/13/developing/querying_basics/chapter_overview.html](Object > Query Interface) to push down relational expressions such as projections, > filtering, sorting, and grouping . > Provided functionality can hopefully address certain Geode use cases and will > provide a stepping stone for future improvements. > Here are some remaining tasks as i see it: > * New tests for test suite (and update calcite-test-dataset to support Geode) > * Add Integration tests that use calcite-test-dataset > * Documentation -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2059) Apache Geode adapter
[ https://issues.apache.org/jira/browse/CALCITE-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355821#comment-16355821 ] Christian Tzolov commented on CALCITE-2059: --- Just rebased and squashed my commits. [~julianhyde] i've been trying to adjust my IDE code style profile to adhere to the Calcite code convention. Obviously i'm still not there :) If you point me to the consistency/stile issues i will try resolve them and improve my IDE's config. But if you prefer to do it yourself please feel free to do it :) [~michaelmior], [~julianhyde] I appreciate your help! > Apache Geode adapter > > > Key: CALCITE-2059 > URL: https://issues.apache.org/jira/browse/CALCITE-2059 > Project: Calcite > Issue Type: New Feature >Reporter: Christian Tzolov >Assignee: Julian Hyde >Priority: Major > > I've been working on a Calcite adapter for [Apache > Geode|http://geode.apache.org]. > Current implementation uses the plain Geode API and > [OQL|http://geode.apache.org/docs/guide/13/developing/querying_basics/chapter_overview.html](Object > Query Interface) to push down relational expressions such as projections, > filtering, sorting, and grouping . > Provided functionality can hopefully address certain Geode use cases and will > provide a stepping stone for future improvements. > Here are some remaining tasks as i see it: > * New tests for test suite (and update calcite-test-dataset to support Geode) > * Add Integration tests that use calcite-test-dataset > * Documentation -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2059) Apache Geode adapter
[ https://issues.apache.org/jira/browse/CALCITE-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355786#comment-16355786 ] Michael Mior commented on CALCITE-2059: --- Thanks for checking [~julianhyde]. I tried to spot these issues but you were obviously more thorough :) > Apache Geode adapter > > > Key: CALCITE-2059 > URL: https://issues.apache.org/jira/browse/CALCITE-2059 > Project: Calcite > Issue Type: New Feature >Reporter: Christian Tzolov >Assignee: Julian Hyde >Priority: Major > > I've been working on a Calcite adapter for [Apache > Geode|http://geode.apache.org]. > Current implementation uses the plain Geode API and > [OQL|http://geode.apache.org/docs/guide/13/developing/querying_basics/chapter_overview.html](Object > Query Interface) to push down relational expressions such as projections, > filtering, sorting, and grouping . > Provided functionality can hopefully address certain Geode use cases and will > provide a stepping stone for future improvements. > Here are some remaining tasks as i see it: > * New tests for test suite (and update calcite-test-dataset to support Geode) > * Add Integration tests that use calcite-test-dataset > * Documentation -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2059) Apache Geode adapter
[ https://issues.apache.org/jira/browse/CALCITE-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355723#comment-16355723 ] Julian Hyde commented on CALCITE-2059: -- I’ve spotted a number of issues (class naming conventions, newlines at ends of files, indentation of method parameters) that are cosmetic bug fixing them is worthwhile because it will improve because it will improve consistency with the rest of calcite. May i make a pass after your rebase? > Apache Geode adapter > > > Key: CALCITE-2059 > URL: https://issues.apache.org/jira/browse/CALCITE-2059 > Project: Calcite > Issue Type: New Feature >Reporter: Christian Tzolov >Assignee: Julian Hyde >Priority: Major > > I've been working on a Calcite adapter for [Apache > Geode|http://geode.apache.org]. > Current implementation uses the plain Geode API and > [OQL|http://geode.apache.org/docs/guide/13/developing/querying_basics/chapter_overview.html](Object > Query Interface) to push down relational expressions such as projections, > filtering, sorting, and grouping . > Provided functionality can hopefully address certain Geode use cases and will > provide a stepping stone for future improvements. > Here are some remaining tasks as i see it: > * New tests for test suite (and update calcite-test-dataset to support Geode) > * Add Integration tests that use calcite-test-dataset > * Documentation -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2059) Apache Geode adapter
[ https://issues.apache.org/jira/browse/CALCITE-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355623#comment-16355623 ] Michael Mior commented on CALCITE-2059: --- [~tzolov] Looks good to me now! If you can rebase the PR into a single commit, I'll proceed with the merge. > Apache Geode adapter > > > Key: CALCITE-2059 > URL: https://issues.apache.org/jira/browse/CALCITE-2059 > Project: Calcite > Issue Type: New Feature >Reporter: Christian Tzolov >Assignee: Julian Hyde >Priority: Major > > I've been working on a Calcite adapter for [Apache > Geode|http://geode.apache.org]. > Current implementation uses the plain Geode API and > [OQL|http://geode.apache.org/docs/guide/13/developing/querying_basics/chapter_overview.html](Object > Query Interface) to push down relational expressions such as projections, > filtering, sorting, and grouping . > Provided functionality can hopefully address certain Geode use cases and will > provide a stepping stone for future improvements. > Here are some remaining tasks as i see it: > * New tests for test suite (and update calcite-test-dataset to support Geode) > * Add Integration tests that use calcite-test-dataset > * Documentation -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2170) Use Druid Expressions capabilities to improve the amount of work that can be pushed to Druid
[
https://issues.apache.org/jira/browse/CALCITE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355497#comment-16355497
]
slim bouguerra commented on CALCITE-2170:
-
[~julianhyde] totally agree with you staying as much as we can in the
Relational realm is the way to go, in fact, I think we are doing that since the
actual DruidQuery is holding an Array of Calcite Rels, and we use those Rels to
generate Druid physical query at the end. Please let me know if you this is not
what you pointed out.
> Use Druid Expressions capabilities to improve the amount of work that can be
> pushed to Druid
>
>
> Key: CALCITE-2170
> URL: https://issues.apache.org/jira/browse/CALCITE-2170
> Project: Calcite
> Issue Type: New Feature
> Components: druid
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
>
> Druid 0.11 has newly built in capabilities called Expressions that can be
> used to push expression like projects/aggregates/filters.
> In order to leverage this new feature, some changes need to be done to the
> Druid Calcite adapter.
> This is a link to the current supported functions and expressions in Druid
> [http://druid.io/docs/latest/misc/math-expr.html]
> As you can see from the Docs an expression can be an actual tree of operators,
> Expression can be used with Filters, Projects, Aggregates, PostAggregates and
> Having filters. For Filters will have new Filter kind called Filter
> expression.
> FYI, you might ask can we push everything as Expression Filter the short
> answer
> is no because, other kinds of Druid filters perform better when used, Hence
> Expression filter is a plan B sort of thing. In order to push expression as
> Projects and Aggregates we will be using Expression based Virtual Columns.
> The major change is the merging of the logic of pushdown verification code and
> the Translation of RexCall/RexNode to Druid Json, native physical language.
> The
> main drive behind this redesign is the fact that in order to check if we can
> push down a tree of expressions to Druid we have to compute the Druid
> Expression
> String anyway. Thus instead of having 2 different code paths, one for pushdown
> validation and one for Json generation we can have one function that does
> both.
> For instance instead of having one code path to test and check if a given
> filter
> can be pushed or not and then having a translation layer code, will have
> one function that either returns a valid Druid Filter or null if it is not
> possible to pushdown. The same idea will be applied to how we push Projects
> and
> Aggregates, Post Aggregates and Sort.
> Here are the main elements/Classes of the new design. First will be merging
> the logic of
> Translation of Literals/InputRex/RexCall to a Druid physical representation.
> Translate leaf RexNode to Valid pair Druid Column + Extraction functions if
> possible
> {code:java}
> /**
> * @param rexNode leaf Input Ref to Druid Column
> * @param rowType row type
> * @param druidQuery druid query
> *
> * @return {@link Pair} of Column name and Extraction Function on the top of
> the input ref or
> * {@link Pair of(null, null)} when can not translate to valid Druid column
> */
> protected static Pair toDruidColumn(RexNode
> rexNode,
> RelDataType rowType, DruidQuery druidQuery
> )
> {code}
> In the other hand, in order to Convert Literals to Druid Literals will
> introduce
> {code:java}
> /**
> * @param rexNode rexNode to translate to Druid literal equivalante
> * @param rowType rowType associated to rexNode
> * @param druidQuery druid Query
> *
> * @return non null string or null if it can not translate to valid Druid
> equivalent
> */
> @Nullable
> private static String toDruidLiteral(RexNode rexNode, RelDataType rowType,
> DruidQuery druidQuery
> )
> {code}
> Main new functions used to pushdown nodes and Druid Json generation.
> Filter pushdown verification and generates is done via
> {code:java}
> org.apache.calcite.adapter.druid.DruidJsonFilter#toDruidFilters
> {code}
> For project pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeProjectAsScan.
> {code}
> For Grouping pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeProjectGroupSet.
> {code}
> For Aggregation pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeDruidJsonAgg
> {code}
> For sort pushdown added
> {code:java}
> org.apache.calcite.adapter.druid.DruidQuery#computeSort\{code}
> Pushing of PostAggregates will be using Expression post Aggregates and use
> {code}
> org.apache.calcite.adapter.druid.DruidExpressions#toDruidExpression\{code}
> to generate expression
> For
[jira] [Commented] (CALCITE-2059) Apache Geode adapter
[ https://issues.apache.org/jira/browse/CALCITE-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355391#comment-16355391 ] Christian Tzolov commented on CALCITE-2059: --- [~michaelmior] thanks for the support. I've made few commits to address the review comments. Also I successfully run the Geode IT tests against a (local) calcite-test-dataset VM. > Apache Geode adapter > > > Key: CALCITE-2059 > URL: https://issues.apache.org/jira/browse/CALCITE-2059 > Project: Calcite > Issue Type: New Feature >Reporter: Christian Tzolov >Assignee: Julian Hyde >Priority: Major > > I've been working on a Calcite adapter for [Apache > Geode|http://geode.apache.org]. > Current implementation uses the plain Geode API and > [OQL|http://geode.apache.org/docs/guide/13/developing/querying_basics/chapter_overview.html](Object > Query Interface) to push down relational expressions such as projections, > filtering, sorting, and grouping . > Provided functionality can hopefully address certain Geode use cases and will > provide a stepping stone for future improvements. > Here are some remaining tasks as i see it: > * New tests for test suite (and update calcite-test-dataset to support Geode) > * Add Integration tests that use calcite-test-dataset > * Documentation -- This message was sent by Atlassian JIRA (v7.6.3#76005)
