[jira] [Updated] (CALCITE-4341) Support double/float mod
[ https://issues.apache.org/jira/browse/CALCITE-4341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunwei Lei updated CALCITE-4341: - Summary: Support double/float mod (was: Calcite support double/float mod) > Support double/float mod > > > Key: CALCITE-4341 > URL: https://issues.apache.org/jira/browse/CALCITE-4341 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jiatao Tao >Assignee: Jiatao Tao >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4332) Improve error when planning rule produces a relational expression with wrong row type
[ https://issues.apache.org/jira/browse/CALCITE-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215201#comment-17215201 ] Chunwei Lei commented on CALCITE-4332: -- I am working on it. > Improve error when planning rule produces a relational expression with wrong > row type > - > > Key: CALCITE-4332 > URL: https://issues.apache.org/jira/browse/CALCITE-4332 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.26.0 >Reporter: Vladimir Sitnikov >Assignee: Chunwei Lei >Priority: Major > > Current code in {{VolcanoPlanner}}: > {code:java} > assert RelOptUtil.equal( > "rel rowtype", > rel.getRowType(), > "equivRel rowtype", > equivRel.getRowType(), > Litmus.THROW); > equivRel = ensureRegistered(equivRel, null); > {code} > The suggested changes: > 1) Use regular error {{IllegalArgumentException}} rather than an assertion > 2) Make sure the exception message contains fine-grained information on what > went wrong. For instance: {{name: non nullable -> nullable; description: > character -> int; ...}} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (CALCITE-4332) Improve error when planning rule produces a relational expression with wrong row type
[ https://issues.apache.org/jira/browse/CALCITE-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunwei Lei reassigned CALCITE-4332: Assignee: Chunwei Lei > Improve error when planning rule produces a relational expression with wrong > row type > - > > Key: CALCITE-4332 > URL: https://issues.apache.org/jira/browse/CALCITE-4332 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.26.0 >Reporter: Vladimir Sitnikov >Assignee: Chunwei Lei >Priority: Major > > Current code in {{VolcanoPlanner}}: > {code:java} > assert RelOptUtil.equal( > "rel rowtype", > rel.getRowType(), > "equivRel rowtype", > equivRel.getRowType(), > Litmus.THROW); > equivRel = ensureRegistered(equivRel, null); > {code} > The suggested changes: > 1) Use regular error {{IllegalArgumentException}} rather than an assertion > 2) Make sure the exception message contains fine-grained information on what > went wrong. For instance: {{name: non nullable -> nullable; description: > character -> int; ...}} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4341) Calcite support double/float mod
[ https://issues.apache.org/jira/browse/CALCITE-4341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiatao Tao updated CALCITE-4341: Summary: Calcite support double/float mod (was: Calcite support double/float mode) > Calcite support double/float mod > > > Key: CALCITE-4341 > URL: https://issues.apache.org/jira/browse/CALCITE-4341 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jiatao Tao >Assignee: Jiatao Tao >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4341) Calcite support double/float mode
Jiatao Tao created CALCITE-4341: --- Summary: Calcite support double/float mode Key: CALCITE-4341 URL: https://issues.apache.org/jira/browse/CALCITE-4341 Project: Calcite Issue Type: Improvement Components: core Reporter: Jiatao Tao Assignee: Jiatao Tao -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3731) Add IF function for BigQuery, Hive and Spark Dialects
[ https://issues.apache.org/jira/browse/CALCITE-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde resolved CALCITE-3731. -- Fix Version/s: 1.27.0 Resolution: Fixed Fixed in [a186f02a|https://github.com/julianhyde/calcite/commit/a186f02ab79374dc7a6ff0b2ae0c3cb661fbd86a]. Thanks for the PR, [~vaibhavjain-dm]! > Add IF function for BigQuery, Hive and Spark Dialects > - > > Key: CALCITE-3731 > URL: https://issues.apache.org/jira/browse/CALCITE-3731 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: vaibhav jain >Assignee: Julian Hyde >Priority: Major > Labels: pull-request-available > Fix For: 1.27.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > Adding support for *IF* function for BigQuery, Hive and Spark Dialects. > *Syntax*- IF(condition, true_result, false_result) > If *condition* is true, returns *true_result*, else returns *false_result*. > Document links: > [BigQuery|https://cloud.google.com/bigquery/docs/reference/standard-sql/conditional_expressions] > [Hive > |https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-ConditionalFunctions] > [Spark Sql|https://spark.apache.org/docs/2.3.1/api/sql/index.html#if] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4335) Aggregate functions for BigQuery
[ https://issues.apache.org/jira/browse/CALCITE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde updated CALCITE-4335: - Description: Add aggregate functions for BigQuery ([spec|https://cloud.google.com/bigquery/docs/reference/standard-sql/aggregate_functions]): {{ARRAY_AGG}}, {{ARRAY_CONCAT_AGG}}, {{COUNTIF}}, {{LOGICAL_AND}}, {{LOGICAL_OR}}, {{STRING_AGG}}. Calcite already has {{ANY_VALUE}}, {{BIT_AND}}, {{BIT_OR}}, {{BIT_XOR}}, {{MAX}}, {{MIN}}, {{SUM}} aggregate functions. (Edit, 2020/10/15: And, since CALCITE-2935 is fixed, it now has {{LOGICAL_AND}}, {{LOGICAL_OR}}.) was: Add aggregate functions for BigQuery ([spec|https://cloud.google.com/bigquery/docs/reference/standard-sql/aggregate_functions]): {{ARRAY_AGG}}, {{ARRAY_CONCAT_AGG}}, {{COUNTIF}}, {{LOGICAL_AND}}, {{LOGICAL_OR}}, {{STRING_AGG}}. Calcite already has {{ANY_VALUE}}, {{BIT_AND}}, {{BIT_OR}}, {{BIT_XOR}}, {{MAX}}, {{MIN}}, {{SUM}} aggregate functions. > Aggregate functions for BigQuery > > > Key: CALCITE-4335 > URL: https://issues.apache.org/jira/browse/CALCITE-4335 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Priority: Major > > Add aggregate functions for BigQuery > ([spec|https://cloud.google.com/bigquery/docs/reference/standard-sql/aggregate_functions]): > {{ARRAY_AGG}}, {{ARRAY_CONCAT_AGG}}, {{COUNTIF}}, {{LOGICAL_AND}}, > {{LOGICAL_OR}}, {{STRING_AGG}}. > Calcite already has {{ANY_VALUE}}, {{BIT_AND}}, {{BIT_OR}}, {{BIT_XOR}}, > {{MAX}}, {{MIN}}, {{SUM}} aggregate functions. > (Edit, 2020/10/15: And, since CALCITE-2935 is fixed, it now has > {{LOGICAL_AND}}, {{LOGICAL_OR}}.) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-2935) Add BOOL_OR, BOOL_AND, LOGICAL_OR, LOGICAL_AND aggregate functions
[ https://issues.apache.org/jira/browse/CALCITE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde resolved CALCITE-2935. -- Fix Version/s: 1.27.0 Resolution: Fixed Fixed in [52eaa919|https://github.com/julianhyde/calcite/commit/52eaa919cea56e3fcce492d7a309f704258e6b5d]. > Add BOOL_OR, BOOL_AND, LOGICAL_OR, LOGICAL_AND aggregate functions > -- > > Key: CALCITE-2935 > URL: https://issues.apache.org/jira/browse/CALCITE-2935 > Project: Calcite > Issue Type: New Feature > Components: core >Reporter: Haisheng Yuan >Assignee: Julian Hyde >Priority: Major > Labels: pull-request-available > Fix For: 1.27.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > Support ANY, SOME, EVERY (also known as BOOL_OR, BOOL_AND) aggregate > functions. > ANY, SOME is equivalent with bool_or. EVERY is equivalent with bool_and. > Parser needs to be changed to support these aggregate functions. > [https://blog.jooq.org/2014/12/18/a-true-sql-gem-you-didnt-know-yet-the-every-aggregate-function/] > > [https://mysqlserverteam.com/using-the-aggregate-functions-any-some-every-with-mysql/] > Note that if ANY or SOME aggregate function is placed on the right side of > comparison operation and argument of this function is a subquery additional > parentheses around aggregate function are required, otherwise it will be > parsed as quantified comparison predicate. > Example: > ANY(NAME LIKE 'W%') > A = (ANY((SELECT B FROM T))) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-2935) Add BOOL_OR, BOOL_AND, LOGICAL_OR, LOGICAL_AND aggregate functions
[ https://issues.apache.org/jira/browse/CALCITE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde updated CALCITE-2935: - Summary: Add BOOL_OR, BOOL_AND, LOGICAL_OR, LOGICAL_AND aggregate functions (was: Add BOOL_OR, BOOL_AND aggregate functions) > Add BOOL_OR, BOOL_AND, LOGICAL_OR, LOGICAL_AND aggregate functions > -- > > Key: CALCITE-2935 > URL: https://issues.apache.org/jira/browse/CALCITE-2935 > Project: Calcite > Issue Type: New Feature > Components: core >Reporter: Haisheng Yuan >Assignee: Julian Hyde >Priority: Major > Labels: pull-request-available > Time Spent: 3.5h > Remaining Estimate: 0h > > Support ANY, SOME, EVERY (also known as BOOL_OR, BOOL_AND) aggregate > functions. > ANY, SOME is equivalent with bool_or. EVERY is equivalent with bool_and. > Parser needs to be changed to support these aggregate functions. > [https://blog.jooq.org/2014/12/18/a-true-sql-gem-you-didnt-know-yet-the-every-aggregate-function/] > > [https://mysqlserverteam.com/using-the-aggregate-functions-any-some-every-with-mysql/] > Note that if ANY or SOME aggregate function is placed on the right side of > comparison operation and argument of this function is a subquery additional > parentheses around aggregate function are required, otherwise it will be > parsed as quantified comparison predicate. > Example: > ANY(NAME LIKE 'W%') > A = (ANY((SELECT B FROM T))) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4340) Correlated Queries in ON clause do not work when referring to left side
[ https://issues.apache.org/jira/browse/CALCITE-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-4340: Labels: pull-request-available (was: ) > Correlated Queries in ON clause do not work when referring to left side > --- > > Key: CALCITE-4340 > URL: https://issues.apache.org/jira/browse/CALCITE-4340 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: James Starr >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Changes from CALCITE-4210 does not robustly support correlated queries in ON > clauses. > For the short term I am going revert the logic for supporting ON clause and > throw an exception when correlated queries occur in ON clause. > Then I will follow up with a fix that will support single subqueries in on > clauses. This will require fixing the offset in Blackboard.register, > correctly detect if correlated variable is on the being used and adding a > rule RelDecorrelator joined decorrelations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4204) Intermittent precision in Druid results when using aggregation functions over columns of type DOUBLE
[ https://issues.apache.org/jira/browse/CALCITE-4204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215025#comment-17215025 ] Julian Hyde edited comment on CALCITE-4204 at 10/15/20, 9:51 PM: - I made progress, and re-enabled quite a few tests, in [ffc1e3b0|https://github.com/apache/calcite/commits/ffc1e3b05e7f920d95c48f7c75fd48372684b8e7], but a few queries remain disabled. was (Author: julianhyde): I made progress, and re-enabled quite a few tests, in [ffc1e3b0|https://github.com/apache/calcite/pull/2212/commits/ffc1e3b05e7f920d95c48f7c75fd48372684b8e7], but a few queries remain disabled. > Intermittent precision in Druid results when using aggregation functions over > columns of type DOUBLE > > > Key: CALCITE-4204 > URL: https://issues.apache.org/jira/browse/CALCITE-4204 > Project: Calcite > Issue Type: Bug > Components: druid-adapter > Environment: The results were obtained by using a dockerized Druid at > 0.17.1: [https://github.com/zabetak/calcite-druid-dataset] >Reporter: Stamatis Zampetakis >Assignee: Julian Hyde >Priority: Major > > Queries with aggregation functions on columns of type DOUBLE return results > where the precision of the columns involved in the computation may differ > from one execution to the other. > Consider the following query which can be found under > DruidAdapterIT#testSingleAverageFunction. > {code:sql} > select "store_state", sum("store_cost") / count(*) as a > from "foodmart" > group by "store_state" > order by a desc > {code} > The same query executed multiple times returns different results > +Result 1+ > {noformat} > store_state=OR; A=2.6271402406293403 > store_state=CA; A=2.599338206292706 > store_state=WA; A=2.582870859286872 > {noformat} > +Result 2+ > {noformat} > store_state=OR; A=2.62714024062934 > store_state=CA; A=2.599338206292706 > store_state=WA; A=2.582870859286872 > {noformat} > +Result 3+ > {noformat} > store_state=OR; A=2.6271402406293394 > store_state=CA; A=2.599338206292706 > store_state=WA; A=2.582870859286872 > {noformat} > Column "store_cost" in Druid is defined as shown below: > {noformat} > "metricsSpec" : [ > ... > { > "name" : "store_sales", > "type" : "doubleSum", > "fieldName" : "store_sales" > } > ...] > {noformat} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-4325) RexSimplify/Sarg incorrectly simplifies complex expressions with NULL
[ https://issues.apache.org/jira/browse/CALCITE-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde resolved CALCITE-4325. -- Fix Version/s: 1.27.0 Resolution: Fixed Fixed in [ee5c7844|https://github.com/apache/calcite/commits/ee5c78444558117c71cc7058a79bc920a52cad59]. > RexSimplify/Sarg incorrectly simplifies complex expressions with NULL > - > > Key: CALCITE-4325 > URL: https://issues.apache.org/jira/browse/CALCITE-4325 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.26.0 > Environment: All environments >Reporter: Igor Lozynskyi >Assignee: Julian Hyde >Priority: Major > Labels: pull-request-available > Fix For: 1.27.0 > > Attachments: rex-simplify-issue-tests-2.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > In Calcite 1.26, the Rex expressions like follows: > {code:java} > (deptno = 20 OR deptno IS NULL) AND deptno = 10{code} > Are simplified to: > {code:java} > deptno IS NULL{code} > Instead of: > {code:java} > FALSE{code} > Similarly, the following expression: > {code:java} > (deptno <> 20 OR deptno IS NULL) AND deptno = 10{code} > Is simplified to: > {code:java} > deptno = 10 OR deptno IS NULL{code} > Instead of: > {code:java} > deptno = 10{code} > A diff with related unit tests (for RelBuilderTest.java) is in the attachment. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4204) Intermittent precision in Druid results when using aggregation functions over columns of type DOUBLE
[ https://issues.apache.org/jira/browse/CALCITE-4204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215025#comment-17215025 ] Julian Hyde commented on CALCITE-4204: -- I made progress, and re-enabled quite a few tests, in [ffc1e3b0|https://github.com/apache/calcite/pull/2212/commits/ffc1e3b05e7f920d95c48f7c75fd48372684b8e7], but a few queries remain disabled. > Intermittent precision in Druid results when using aggregation functions over > columns of type DOUBLE > > > Key: CALCITE-4204 > URL: https://issues.apache.org/jira/browse/CALCITE-4204 > Project: Calcite > Issue Type: Bug > Components: druid-adapter > Environment: The results were obtained by using a dockerized Druid at > 0.17.1: [https://github.com/zabetak/calcite-druid-dataset] >Reporter: Stamatis Zampetakis >Assignee: Julian Hyde >Priority: Major > > Queries with aggregation functions on columns of type DOUBLE return results > where the precision of the columns involved in the computation may differ > from one execution to the other. > Consider the following query which can be found under > DruidAdapterIT#testSingleAverageFunction. > {code:sql} > select "store_state", sum("store_cost") / count(*) as a > from "foodmart" > group by "store_state" > order by a desc > {code} > The same query executed multiple times returns different results > +Result 1+ > {noformat} > store_state=OR; A=2.6271402406293403 > store_state=CA; A=2.599338206292706 > store_state=WA; A=2.582870859286872 > {noformat} > +Result 2+ > {noformat} > store_state=OR; A=2.62714024062934 > store_state=CA; A=2.599338206292706 > store_state=WA; A=2.582870859286872 > {noformat} > +Result 3+ > {noformat} > store_state=OR; A=2.6271402406293394 > store_state=CA; A=2.599338206292706 > store_state=WA; A=2.582870859286872 > {noformat} > Column "store_cost" in Druid is defined as shown below: > {noformat} > "metricsSpec" : [ > ... > { > "name" : "store_sales", > "type" : "doubleSum", > "fieldName" : "store_sales" > } > ...] > {noformat} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4340) Correlated Queries in ON clause do not work when referring to left side
James Starr created CALCITE-4340: Summary: Correlated Queries in ON clause do not work when referring to left side Key: CALCITE-4340 URL: https://issues.apache.org/jira/browse/CALCITE-4340 Project: Calcite Issue Type: Improvement Components: core Reporter: James Starr Changes from CALCITE-4210 does not robustly support correlated queries in ON clauses. For the short term I am going revert the logic for supporting ON clause and throw an exception when correlated queries occur in ON clause. Then I will follow up with a fix that will support single subqueries in on clauses. This will require fixing the offset in Blackboard.register, correctly detect if correlated variable is on the being used and adding a rule RelDecorrelator joined decorrelations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4293) cassandra adapter returns null when selecting non-null tuple elements
[ https://issues.apache.org/jira/browse/CALCITE-4293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17214982#comment-17214982 ] Alessandro Solimando edited comment on CALCITE-4293 at 10/15/20, 8:28 PM: -- As suggested I have checked in Postgres, it's indeed forbidden to try to access a Record via the ITEM operator. The following query: {noformat} select "T"."X"[1] from (VALUES (ROW(ROW(1,2), ROW(4, 8 as "T"("X") {noformat} Gives this error: {noformat} ERROR: cannot subscript type record because it is not an array SQL state: 42804{noformat} All this looks like well [documented|https://www.postgresql.org/docs/9.1/sql-expressions.html] (Section 4.2.3. Subscripts), and it conforms to the SQL standard, as per Stamatis' first reply, so it all adds up fine. As Julian suggested, in principle it should not be possible to apply `ITEM` to a Struct, so the query mentioned in the ticket description should fail. However, at [SqlItemOperator#96|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlItemOperator.java#L96] we can see that we support access via a "String"-typed argument when `ITEM` is applied to a Struct/Row (same goes for the rest, the type checker accepts to apply `ITEM` on `ANY` type etc., with the notable exception of the “getAllowedSignatures” method, which is outdated). See also [SqlValidatorTest.java#L11933,|https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/SqlValidatorTest.java#L11933] which uses the ITEM operator over a Struct. Now the remaining question is: do we want to support positional access for “anonymous” struct types too? IMO this extension is in line with the partial support for applying ITEM to Struct types with no particular drawbacks, and it’s less surprising than the existing partial support we have right now (I got fooled by that when writing the unit tests for Cassandra, for instance, since the type system was not complaining I assumed it was OK). If this makes sense to you as well, I propose to open a new ticket and keep discussing there for those who are interested. was (Author: asolimando): As suggested I have checked in Postgres, it's indeed forbidden to try to access a Record by index. The following query: {noformat} select "T"."X"[1] from (VALUES (ROW(ROW(1,2), ROW(4, 8 as "T"("X") {noformat} Gives this error: {noformat} ERROR: cannot subscript type record because it is not an array SQL state: 42804{noformat} All this looks like well [documented|https://www.postgresql.org/docs/9.1/sql-expressions.html] (Section 4.2.3. Subscripts), and it conforms to the SQL standard, as per Stamatis' first reply, so it all adds up fine. As Julian suggested, in principle it should not be possible to apply `ITEM` to a Struct, so the query mentioned in the ticket description should fail. However, at [SqlItemOperator#96|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlItemOperator.java#L96] we can see that we support access via a "String"-typed argument when `ITEM` is applied to a Struct/Row (same goes for the rest, the type checker accepts to apply `ITEM` on `ANY` type etc., with the notable exception of the “getAllowedSignatures” method, which is outdated). See also [SqlValidatorTest.java#L11933,|https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/SqlValidatorTest.java#L11933] which uses the ITEM operator over a Struct. Now the remaining question is: do we want to support positional access for “anonymous” struct types too? IMO this extension is in line with the partial support for applying ITEM to Struct types with no particular drawbacks, and it’s less surprising than the existing partial support we have right now (I got fooled by that when writing the unit tests for Cassandra, for instance, since the type system was not complaining I assumed it was OK). If this makes sense to you as well, I propose to open a new ticket and keep discussing there for those who are interested. > cassandra adapter returns null when selecting non-null tuple elements > - > > Key: CALCITE-4293 > URL: https://issues.apache.org/jira/browse/CALCITE-4293 > Project: Calcite > Issue Type: Bug > Components: cassandra-adapter >Affects Versions: 1.25.0 >Reporter: Alessandro Solimando >Assignee: Alessandro Solimando >Priority: Major > > The following test currently fails due to the _EXPR$i_ elements are null and > don't match their actual value within the _f_tuple_ field: > {code:java} > @Test void testTupleInnerValues() { > CalciteAssert.that() > .with(DTCASSANDRA) > .query("select x['1'], x['2'], x['3'] from " > +
[jira] [Comment Edited] (CALCITE-4293) cassandra adapter returns null when selecting non-null tuple elements
[ https://issues.apache.org/jira/browse/CALCITE-4293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17214982#comment-17214982 ] Alessandro Solimando edited comment on CALCITE-4293 at 10/15/20, 8:24 PM: -- As suggested I have checked in Postgres, it's indeed forbidden to try to access a Record by index. The following query: {noformat} select "T"."X"[1] from (VALUES (ROW(ROW(1,2), ROW(4, 8 as "T"("X") {noformat} Gives this error: {noformat} ERROR: cannot subscript type record because it is not an array SQL state: 42804{noformat} All this looks like well [documented|https://www.postgresql.org/docs/9.1/sql-expressions.html] (Section 4.2.3. Subscripts), and it conforms to the SQL standard, as per Stamatis' first reply, so it all adds up fine. As Julian suggested, in principle it should not be possible to apply `ITEM` to a Struct, so the query mentioned in the ticket description should fail. However, at [SqlItemOperator#96|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlItemOperator.java#L96] we can see that we support access via a "String"-typed argument when `ITEM` is applied to a Struct/Row (same goes for the rest, the type checker accepts to apply `ITEM` on `ANY` type etc., with the notable exception of the “getAllowedSignatures” method, which is outdated). See also [SqlValidatorTest.java#L11933,|https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/SqlValidatorTest.java#L11933] which uses the ITEM operator over a Struct. Now the remaining question is: do we want to support positional access for “anonymous” struct types too? IMO this extension is in line with the partial support for applying ITEM to Struct types with no particular drawbacks, and it’s less surprising than the existing partial support we have right now (I got fooled by that when writing the unit tests for Cassandra, for instance, since the type system was not complaining I assumed it was OK). If this makes sense to you as well, I propose to open a new ticket and keep discussing there for those who are interested. was (Author: asolimando): As suggested I have checked in Postgres, it's indeed forbidden to try to access a Record by index. The following query: {noformat} select "T"."X"[1] from (VALUES (ROW(ROW(1,2), ROW(4, 8 as "T"("X") {noformat} Gives this error: {noformat} ERROR: cannot subscript type record because it is not an array SQL state: 42804{noformat} All this looks like well [documented|https://www.postgresql.org/docs/9.1/sql-expressions.html] (Section 4.2.3. Subscripts), and it conforms to the SQL standard, as per Stamatis' first reply, so it all adds up fine. As Julian suggested, in principle it should not be possible to apply `ITEM` to a Struct, so the query mentioned in the ticket description should fail. However, at [SqlItemOperator#96|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlItemOperator.java#L96] we can see that we support access via a "String"-typed argument when `ITEM` is applied to a Struct/Row (same goes for the rest, the type checker accepts to apply `ITEM` on `ANY` type etc., with the notable exception of the “getAllowedSignatures” method, which is outdated). See also [SqlValidatorTest.java#L11933,|https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/SqlValidatorTest.java#L11933] which uses the ITEM operator over a Struct. Now the remaining question is: do we want to support positional access for “anonymous” struct types too? IMO this extension is in line with the partial support for applying ITEM to Struct types with no particular drawbacks, and it’s less surprising than the existing partial support we have right now (I got fooled by that when writing the unit tests for Cassandra, for instance, since the type system was not complaining I assumed it was OK). If this makes sense to you as well, I propose to open a new ticket and keep discussing there for those who are interested. > cassandra adapter returns null when selecting non-null tuple elements > - > > Key: CALCITE-4293 > URL: https://issues.apache.org/jira/browse/CALCITE-4293 > Project: Calcite > Issue Type: Bug > Components: cassandra-adapter >Affects Versions: 1.25.0 >Reporter: Alessandro Solimando >Assignee: Alessandro Solimando >Priority: Major > > The following test currently fails due to the _EXPR$i_ elements are null and > don't match their actual value within the _f_tuple_ field: > {code:java} > @Test void testTupleInnerValues() { > CalciteAssert.that() > .with(DTCASSANDRA) > .query("select x['1'], x['2'], x['3'] from " > + "(select
[jira] [Commented] (CALCITE-4293) cassandra adapter returns null when selecting non-null tuple elements
[ https://issues.apache.org/jira/browse/CALCITE-4293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17214982#comment-17214982 ] Alessandro Solimando commented on CALCITE-4293: --- As suggested I have checked in Postgres, it's indeed forbidden to try to access a Record by index. The following query: {noformat} select "T"."X"[1] from (VALUES (ROW(ROW(1,2), ROW(4, 8 as "T"("X") {noformat} Gives this error: {noformat} ERROR: cannot subscript type record because it is not an array SQL state: 42804{noformat} All this looks like well [documented|https://www.postgresql.org/docs/9.1/sql-expressions.html] (Section 4.2.3. Subscripts), and it conforms to the SQL standard, as per Stamatis' first reply, so it all adds up fine. As Julian suggested, in principle it should not be possible to apply `ITEM` to a Struct, so the query mentioned in the ticket description should fail. However, at [SqlItemOperator#96|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlItemOperator.java#L96] we can see that we support access via a "String"-typed argument when `ITEM` is applied to a Struct/Row (same goes for the rest, the type checker accepts to apply `ITEM` on `ANY` type etc., with the notable exception of the “getAllowedSignatures” method, which is outdated). See also [SqlValidatorTest.java#L11933,|https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/SqlValidatorTest.java#L11933] which uses the ITEM operator over a Struct. Now the remaining question is: do we want to support positional access for “anonymous” struct types too? IMO this extension is in line with the partial support for applying ITEM to Struct types with no particular drawbacks, and it’s less surprising than the existing partial support we have right now (I got fooled by that when writing the unit tests for Cassandra, for instance, since the type system was not complaining I assumed it was OK). If this makes sense to you as well, I propose to open a new ticket and keep discussing there for those who are interested. > cassandra adapter returns null when selecting non-null tuple elements > - > > Key: CALCITE-4293 > URL: https://issues.apache.org/jira/browse/CALCITE-4293 > Project: Calcite > Issue Type: Bug > Components: cassandra-adapter >Affects Versions: 1.25.0 >Reporter: Alessandro Solimando >Assignee: Alessandro Solimando >Priority: Major > > The following test currently fails due to the _EXPR$i_ elements are null and > don't match their actual value within the _f_tuple_ field: > {code:java} > @Test void testTupleInnerValues() { > CalciteAssert.that() > .with(DTCASSANDRA) > .query("select x['1'], x['2'], x['3'] from " > + "(select \"f_tuple\" from \"test_collections\") as T(x)") > .returns("EXPR$0=30" > + "; EXPR$1=30ff87" > + "; EXPR$2=2015-05-03 13:30:54\n"); > }{code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4317) Some rules fail to handle Aggregate node if RelFieldTrimmer trims all the fields
[ https://issues.apache.org/jira/browse/CALCITE-4317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17214942#comment-17214942 ] Rafay A commented on CALCITE-4317: -- [~julianhyde], one other example is what if the incoming RelNode has no columns, prior to trimming, e.g. {code:java} select 1 as col from sales.customer group by (){code} will generate a RelNode: {code:java} LogicalProject(COL=[1]) LogicalAggregate(group=[{}]) LogicalProject(DUMMY=[0]) LogicalTableScan(table=[[CATALOG, SALES, CUSTOMER]]){code} If we trim this by applying the above approach (adding a dummy COUNT() in the LogicalAggregate) will not work since that will generate a row type of size 1 whereas the parent trimFieldsDispatcher expects a row size of zero. Should the trimmer suppose to handle these cases as well? > Some rules fail to handle Aggregate node if RelFieldTrimmer trims all the > fields > > > Key: CALCITE-4317 > URL: https://issues.apache.org/jira/browse/CALCITE-4317 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Rafay A >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Consider this query: > {code:java} > select o_orderkey from > (SELECT count(*) cnt_star, count(l_orderkey) cnt_ok FROM lineitem) > cross join orders > limit 10 > {code} > which generates this plan: > {code:java} > LogicalSort(fetch=[10]) > LogicalProject(o_orderkey=[$2]) > LogicalJoin(condition=[true], joinType=[inner]) > LogicalAggregate(group=[{}], cnt_star=[COUNT()], cnt_ok=[COUNT($0)]) > LogicalProject(l_orderkey=[$0]) > ScanCrel(table=[lineitem], columns=[`l_orderkey`, `l_partkey`, > `l_suppkey`, `l_linenumber`, `l_quantity`, `l_extendedprice`, `l_discount`, > `l_tax`, `l_returnflag`, `l_linestatus`, `l_shipdate`, `l_commitdate`, > `l_receiptdate`, `l_shipinstruct`, `l_shipmode`, `l_comment`]) > ScanCrel(table=[orders], columns=[`o_orderkey`, `o_custkey`, > `o_orderstatus`, `o_totalprice`, `o_orderdate`, `o_orderpriority`, `o_clerk`, > `o_shippriority`, `o_comment`]) > {code} > and after we apply RelFieldTrimmer: > {code:java} > LogicalSort(fetch=[10]) > LogicalProject(o_orderkey=[$0]) > LogicalJoin(condition=[true], joinType=[inner]) > LogicalAggregate(group=[{}]) > LogicalProject(l_orderkey=[$0]) > ScanCrel(table=[lineitem], columns=[`l_orderkey`]) > ScanCrel(table=[orders], columns=[`o_orderkey`]) > {code} > > We see that the LogicalAggregate has no groups/functions after applying the > trimmer. When we apply ProjectJoinTransposeRule or LoptOptimizeJoinRule after > trimming, we are seeing these exceptions: > For LoptOptimizeJoinRule: > {code:java} > (java.lang.ArrayIndexOutOfBoundsException) 0 > com.google.common.collect.RegularImmutableList.get():75 > org.apache.calcite.rel.metadata.RelMdColumnOrigins.getColumnOrigins():77 > sun.reflect.NativeMethodAccessorImpl.invoke0():-2 > sun.reflect.NativeMethodAccessorImpl.invoke():62 > sun.reflect.DelegatingMethodAccessorImpl.invoke():43 > java.lang.reflect.Method.invoke():498 > > org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$1$1.invoke():178 > com.sun.proxy.$Proxy175.getColumnOrigins():-1 > sun.reflect.GeneratedMethodAccessor144.invoke():-1 > sun.reflect.DelegatingMethodAccessorImpl.invoke():43 > java.lang.reflect.Method.invoke():498 > > org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke():139 > com.sun.proxy.$Proxy175.getColumnOrigins():-1 > sun.reflect.GeneratedMethodAccessor144.invoke():-1 > sun.reflect.DelegatingMethodAccessorImpl.invoke():43 > java.lang.reflect.Method.invoke():498 > com.sun.proxy.$Proxy175.getColumnOrigins():-1 > sun.reflect.GeneratedMethodAccessor144.invoke():-1 > sun.reflect.DelegatingMethodAccessorImpl.invoke():43 > java.lang.reflect.Method.invoke():498 > > org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke():139 > com.sun.proxy.$Proxy175.getColumnOrigins():-1 > sun.reflect.GeneratedMethodAccessor144.invoke():-1 > sun.reflect.DelegatingMethodAccessorImpl.invoke():43 > java.lang.reflect.Method.invoke():498 > com.sun.proxy.$Proxy175.getColumnOrigins():-1 > sun.reflect.GeneratedMethodAccessor144.invoke():-1 > sun.reflect.DelegatingMethodAccessorImpl.invoke():43 > java.lang.reflect.Method.invoke():498 > > org.apache.calcite.rel.metadata.RelMetadataQuery$MetadataInvocationHandler.invoke():110 > com.sun.proxy.$Proxy175.getColumnOrigins():-1 > org.apache.calcite.rel.metadata.RelMetadataQuery.getColumnOrigins():293 >
[jira] [Commented] (CALCITE-4290) Not optimal subqueries due to a "*" in them
[ https://issues.apache.org/jira/browse/CALCITE-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17214727#comment-17214727 ] Stamatis Zampetakis commented on CALCITE-4290: -- Hey [~rr111], you may find something useful in the following links: https://lists.apache.org/thread.html/re6bb7e4db4b71296dba164ca2784efdec5b925105af0aa8f02cd3f5f%40%3Cdev.calcite.apache.org%3E https://lists.apache.org/thread.html/4d4019b01a9dec08467204baaf0a50f8f516241357a3e881fdde022e%40%3Cdev.calcite.apache.org%3E https://github.com/zabetak/calcite/commit/c7646fa6d3054b9e6b9a800e08d18f3cc89922e4 > Not optimal subqueries due to a "*" in them > --- > > Key: CALCITE-4290 > URL: https://issues.apache.org/jira/browse/CALCITE-4290 > Project: Calcite > Issue Type: Bug > Components: jdbc-adapter >Affects Versions: 1.25.0 >Reporter: Roman Churganov >Priority: Major > > Run a query which should be distributed into sub-queries onto multiple > schemas through JDBC adapter, for example > select ft.id, ft.c11, tt.c41 from sch1.foo ft join sch2.tab tt on > ft.id = tt.id > Calcite will make two sub-queries like SELECT * FROM "TAB" ORDER BY "ID" and > SELECT * FROM "FOO" ORDER BY "ID" which are not optimal due to an excessive > columns data requested > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4292) Remove exist condition from Not equals criteria
[ https://issues.apache.org/jira/browse/CALCITE-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-4292: Labels: ElasticSearch NotEquals QueryBuilder calcite pull-request-available (was: ElasticSearch NotEquals QueryBuilder calcite) > Remove exist condition from Not equals criteria > --- > > Key: CALCITE-4292 > URL: https://issues.apache.org/jira/browse/CALCITE-4292 > Project: Calcite > Issue Type: Bug > Components: elasticsearch-adapter >Reporter: Shlok Srivastava >Assignee: Bill Neil >Priority: Major > Labels: ElasticSearch, NotEquals, QueryBuilder, calcite, > pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Remove the exists condition from Not equals Query Expression. Elasticsearch > doesn't put this condition therefore keeping queries in sync. > [Code|https://github.com/apache/calcite/blob/1050b36cafbb0c487b7a2ade3efd12850609717e/elasticsearch/src/main/java/org/apache/calcite/adapter/elasticsearch/PredicateAnalyzer.java#L782] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4323) View with ORDER BY throws AssertionError during view expansion
[ https://issues.apache.org/jira/browse/CALCITE-4323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17214482#comment-17214482 ] Jiatao Tao edited comment on CALCITE-4323 at 10/15/20, 8:25 AM: OK, I'll pull my request. Note: need to deal # sub-query in view # view as a sub-query was (Author: aron.tao): OK, I'll pull my request. > View with ORDER BY throws AssertionError during view expansion > -- > > Key: CALCITE-4323 > URL: https://issues.apache.org/jira/browse/CALCITE-4323 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Jiatao Tao >Priority: Major > Attachments: image-2020-10-10-23-58-39-822.png, > image-2020-10-10-23-59-22-297.png > > > If you have a view that has an ORDER BY clause, and try to execute a query on > that view, SqlToRelConverter throws AssertionError during view expansion. For > example, > {noformat} > create view v as select * from "EMPLOYEES" order by "deptno"; > select * from V; > java.lang.AssertionError > at > org.apache.calcite.sql2rel.SqlToRelConverter.requiredCollation(SqlToRelConverter.java:634) > at > org.apache.calcite.sql2rel.SqlToRelConverter.requiredCollation(SqlToRelConverter.java:629) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:578) > at > org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.expandView(CalcitePrepareImpl.java:1072) > at > org.apache.calcite.plan.ViewExpanders$1.expandView(ViewExpanders.java:52) > at > org.apache.calcite.schema.impl.ViewTable.expandView(ViewTable.java:127) > at org.apache.calcite.schema.impl.ViewTable.toRel(ViewTable.java:120) > at > org.apache.calcite.prepare.RelOptTableImpl.toRel(RelOptTableImpl.java:285) > at > org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:3605) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2522) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2160) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2109) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2066) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:662) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:643) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3458) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:569) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:242) > {noformat} > I suspected that it would be a problem for when the view does not project the > sort column(s), but it seems to be a problem even if the columns are > projected. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4323) View with ORDER BY throws AssertionError during view expansion
[ https://issues.apache.org/jira/browse/CALCITE-4323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17214482#comment-17214482 ] Jiatao Tao commented on CALCITE-4323: - OK, I'll pull my request. > View with ORDER BY throws AssertionError during view expansion > -- > > Key: CALCITE-4323 > URL: https://issues.apache.org/jira/browse/CALCITE-4323 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Jiatao Tao >Priority: Major > Attachments: image-2020-10-10-23-58-39-822.png, > image-2020-10-10-23-59-22-297.png > > > If you have a view that has an ORDER BY clause, and try to execute a query on > that view, SqlToRelConverter throws AssertionError during view expansion. For > example, > {noformat} > create view v as select * from "EMPLOYEES" order by "deptno"; > select * from V; > java.lang.AssertionError > at > org.apache.calcite.sql2rel.SqlToRelConverter.requiredCollation(SqlToRelConverter.java:634) > at > org.apache.calcite.sql2rel.SqlToRelConverter.requiredCollation(SqlToRelConverter.java:629) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:578) > at > org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.expandView(CalcitePrepareImpl.java:1072) > at > org.apache.calcite.plan.ViewExpanders$1.expandView(ViewExpanders.java:52) > at > org.apache.calcite.schema.impl.ViewTable.expandView(ViewTable.java:127) > at org.apache.calcite.schema.impl.ViewTable.toRel(ViewTable.java:120) > at > org.apache.calcite.prepare.RelOptTableImpl.toRel(RelOptTableImpl.java:285) > at > org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:3605) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2522) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2160) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2109) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2066) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:662) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:643) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3458) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:569) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:242) > {noformat} > I suspected that it would be a problem for when the view does not project the > sort column(s), but it seems to be a problem even if the columns are > projected. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4339) Update Gradle: 6.6 -> 6.7
[ https://issues.apache.org/jira/browse/CALCITE-4339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-4339: Labels: pull-request-available (was: ) > Update Gradle: 6.6 -> 6.7 > - > > Key: CALCITE-4339 > URL: https://issues.apache.org/jira/browse/CALCITE-4339 > Project: Calcite > Issue Type: Improvement > Components: build >Affects Versions: 1.26.0 >Reporter: Vladimir Sitnikov >Assignee: Vladimir Sitnikov >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4339) Update Gradle: 6.6 -> 6.7
Vladimir Sitnikov created CALCITE-4339: -- Summary: Update Gradle: 6.6 -> 6.7 Key: CALCITE-4339 URL: https://issues.apache.org/jira/browse/CALCITE-4339 Project: Calcite Issue Type: Improvement Components: build Affects Versions: 1.26.0 Reporter: Vladimir Sitnikov Assignee: Vladimir Sitnikov -- This message was sent by Atlassian Jira (v8.3.4#803005)