[jira] [Assigned] (CALCITE-3368) Nonequivalent simplification happens in RexSimplify
[
https://issues.apache.org/jira/browse/CALCITE-3368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan reassigned CALCITE-3368:
---
Assignee: Danny Chan
> Nonequivalent simplification happens in RexSimplify
> ---
>
> Key: CALCITE-3368
> URL: https://issues.apache.org/jira/browse/CALCITE-3368
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
>Reporter: Leonard Xu
>Assignee: Danny Chan
>Priority: Major
>
> when I test a Fink sql to check overflow situation like
>
> {code:java}
> select
>case when (f0 + f1) is null then 'null' else 'not null' end
> from testTable
> {code}
> , I found the ' when (f0 + f1) is null' has been simplified by Calcite, and
> the simplification may be incorrect.
>
> The Calcite's simplification logic of isNull expression in SQL will convert
> from
> *"f(operand0, operand1) IS NULL"* to
> *"operand0 IS NULL OR operand1 IS NULL"* if the Policy of RexNode‘s SqlKind
> is ANY。
> This simplification leads to the expression will not calculate the real
> value of *f(operand0, operand1)* (eg..'f0 + 'f1 ）which maybe overflow.
> {code:java}
> //org.apache.calcite.rex.RexSimplify.java
> private RexNode simplifyIsNull(RexNode a) {
> // Simplify the argument first,
> // call ourselves recursively to see whether we can make more progress.
> // For example, given
> // "(CASE WHEN FALSE THEN 1 ELSE 2) IS NULL" we first simplify the
> // argument to "2", and only then we can simplify "2 IS NULL" to "FALSE".
> a = simplify(a, UNKNOWN);
> if (!a.getType().isNullable() && isSafeExpression(a)) {
> return rexBuilder.makeLiteral(false);
> }
> if (RexUtil.isNull(a)) {
> return rexBuilder.makeLiteral(true);
> }
> if (a.getKind() == SqlKind.CAST) {
> return null;
> }
> switch (Strong.policy(a.getKind())) {
> case NOT_NULL:
> return rexBuilder.makeLiteral(false);
> case ANY:
> // "f" is a strong operator, so "f(operand0, operand1) IS NULL" simplifies
> // to "operand0 IS NULL OR operand1 IS NULL"
> final List operands = new ArrayList<>();
> for (RexNode operand : ((RexCall) a).getOperands()) {
> final RexNode simplified = simplifyIsNull(operand);
> if (simplified == null) {
> operands.add(
> rexBuilder.makeCall(SqlStdOperatorTable.IS_NULL, operand));
> } else {
> operands.add(simplified);
> }
> }
> return RexUtil.composeDisjunction(rexBuilder, operands, false);
> case AS_IS:
> default:
> return null;
> }
> }{code}
> And most of calculating SqlKinds are assigned *Policy.ANY* at present.
> {code:java}
> //org.apache.calcite.plan.Strong.java
> public static Policy policy(SqlKind kind) {
> return MAP.getOrDefault(kind, Policy.AS_IS);
> }
>
> map.put(SqlKind.PLUS, Policy.ANY);
> map.put(SqlKind.PLUS_PREFIX, Policy.ANY);
> map.put(SqlKind.MINUS, Policy.ANY);
> map.put(SqlKind.MINUS_PREFIX, Policy.ANY);
> map.put(SqlKind.TIMES, Policy.ANY);
> map.put(SqlKind.DIVIDE, Policy.ANY);
> * that operator evaluates to null. */
> public enum Policy {
> /** This kind of expression is never null. No need to look at its arguments,
>* if it has any. */
> NOT_NULL,
> /** This kind of expression has its own particular rules about whether it
>* is null. */
> CUSTOM,
> /** This kind of expression is null if and only if at least one of its
>* arguments is null. */
> ANY,
> /** This kind of expression may be null. There is no way to rewrite. */
> AS_IS,
> }{code}
>
> It may be an obvious nonequivalent simplification in SQL.
> And this issue come from Flink （FLINK-14030）.
>
> [~danny0405]， Could you have a look at this?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3159) Distinct can be removed for MIN/MAX/BIT_OR/BIT_AND aggregate functions
[
https://issues.apache.org/jira/browse/CALCITE-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3159.
-
Assignee: Danny Chan (was: Julian Hyde)
Resolution: Fixed
Fixed in
[3abb974|https://github.com/apache/calcite/commit/3abb974588194e2e531aa43daa79c4dfd18ef25f],
thanks for your PR, [~x1q1j1] !
> Distinct can be removed for MIN/MAX/BIT_OR/BIT_AND aggregate functions
> --
>
> Key: CALCITE-3159
> URL: https://issues.apache.org/jira/browse/CALCITE-3159
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Haisheng Yuan
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> For the following query:
> {code:java}
> select a, min(distinct b), bit_or(distinct c) from foo group by a;
> {code}
> Currently Calcite still preserve the distinct for these aggregate functions,
> but DISTINCT is not meaningful with MIN/MAX and is available for ISO
> compatibility only. We can safely remove distinct and get more optimization
> opportunities.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3159) Distinct can be removed for MIN/MAX/BIT_OR/BIT_AND aggregate functions
[
https://issues.apache.org/jira/browse/CALCITE-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3159:
Fix Version/s: 1.21.0
> Distinct can be removed for MIN/MAX/BIT_OR/BIT_AND aggregate functions
> --
>
> Key: CALCITE-3159
> URL: https://issues.apache.org/jira/browse/CALCITE-3159
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Haisheng Yuan
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> For the following query:
> {code:java}
> select a, min(distinct b), bit_or(distinct c) from foo group by a;
> {code}
> Currently Calcite still preserve the distinct for these aggregate functions,
> but DISTINCT is not meaningful with MIN/MAX and is available for ISO
> compatibility only. We can safely remove distinct and get more optimization
> opportunities.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3159) Distinct can be removed for MIN/MAX/BIT_OR/BIT_AND aggregate functions
[
https://issues.apache.org/jira/browse/CALCITE-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3159:
Affects Version/s: 1.20.0
> Distinct can be removed for MIN/MAX/BIT_OR/BIT_AND aggregate functions
> --
>
> Key: CALCITE-3159
> URL: https://issues.apache.org/jira/browse/CALCITE-3159
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.20.0
>Reporter: Haisheng Yuan
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> For the following query:
> {code:java}
> select a, min(distinct b), bit_or(distinct c) from foo group by a;
> {code}
> Currently Calcite still preserve the distinct for these aggregate functions,
> but DISTINCT is not meaningful with MIN/MAX and is available for ISO
> compatibility only. We can safely remove distinct and get more optimization
> opportunities.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3364) Can't group table function result due to a type cast error if table function returns a row with a single value
[
https://issues.apache.org/jira/browse/CALCITE-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935810#comment-16935810
]
Danny Chan commented on CALCITE-3364:
-
Well, seems we should fix this problem in the linq4j, i would like to take some
time to dig into this problem in the following days.
> Can't group table function result due to a type cast error if table function
> returns a row with a single value
> --
>
> Key: CALCITE-3364
> URL: https://issues.apache.org/jira/browse/CALCITE-3364
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Anton Haidai
>Priority: Major
>
> I was not able to find a suitable test, so I'll describe the issue using a
> custom minimal table function returning a row with a single value as an
> example. I believe, that it should be reproducible against any table function.
> There is a simple table function my_dummy() that just prints numbers 1 to 5.
> Simple select works as expected:
> {code}
> SQL:
> select val from table(my_dummy())
> Result:
> val : INTEGER
> 5
> 4
> 3
> 2
> 1
> {code}
> However, when trying to make an aggregation on top of it, there is a class
> cast error:
> {code}
> SQL:
> select val from table(my_dummy())
> group by val
> Error:
> java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to
> java.lang.Integer
> at
> org.apache.calcite.avatica.util.AbstractCursor$IntAccessor.getInt(AbstractCursor.java:541)
> at org.apache.calcite.avatica.AvaticaSite.get(AvaticaSite.java:340)
> at
> org.apache.calcite.avatica.AvaticaResultSet.getObject(AvaticaResultSet.java:393)
> {code}
> Actually, this array Object[] contains a single integer produced by the table
> function. Also I faced similar class cast errors in a generated code making
> hashJoin (in a complex production code), I believe, the cause should be the
> same as in this minimal example.
> Here is this my_dummy() table function implementation (single file):
> {code}
> package com.myapp.tf;
> import org.apache.calcite.DataContext;
> import org.apache.calcite.linq4j.AbstractEnumerable;
> import org.apache.calcite.linq4j.Enumerable;
> import org.apache.calcite.linq4j.Enumerator;
> import org.apache.calcite.linq4j.tree.Types;
> import org.apache.calcite.rel.type.RelDataType;
> import org.apache.calcite.rel.type.RelDataTypeFactory;
> import org.apache.calcite.schema.ScannableTable;
> import org.apache.calcite.schema.TableFunction;
> import org.apache.calcite.schema.impl.AbstractTable;
> import org.apache.calcite.schema.impl.TableFunctionImpl;
> import org.apache.calcite.sql.SqlIdentifier;
> import org.apache.calcite.sql.SqlOperatorBinding;
> import org.apache.calcite.sql.parser.SqlParserPos;
> import org.apache.calcite.sql.type.SqlTypeName;
> import org.apache.calcite.sql.validate.SqlUserDefinedTableFunction;
> import java.util.Collections;
> import java.util.concurrent.atomic.AtomicInteger;
> import static org.apache.calcite.sql.type.OperandTypes.family;
> public class DummyFunction extends SqlUserDefinedTableFunction {
> public static String DUMMY_FUNCTION_NAME = "my_dummy";
> public static final TableFunction DUMMY_TABLE_FUNCTION =
> TableFunctionImpl.create(Types.lookupMethod(
> DummyFunction.class,
> "createDummyTable"
> ));
> public static DummyTable createDummyTable() {
> return new DummyTable();
> }
> public DummyFunction() {
> super(new SqlIdentifier(DUMMY_FUNCTION_NAME, SqlParserPos.ZERO),
> null,
> null,
> family(),
> Collections.emptyList(),
> DUMMY_TABLE_FUNCTION
> );
> }
> @Override
> public RelDataType inferReturnType(SqlOperatorBinding opBinding) {
> RelDataTypeFactory typeFactory = opBinding.getTypeFactory();
> return typeFactory.builder()
> .add("val", SqlTypeName.INTEGER)
> .build();
> }
> }
> class DummyTable extends AbstractTable implements ScannableTable {
> private final AtomicInteger cnt = new AtomicInteger(6);
> @Override
> public Enumerable scan(DataContext root) {
> return new AbstractEnumerable() {
> public Enumerator enumerator() {
> return new Enumerator () {
> @Override
> public Object[] current() {
> return new Object[] { cnt.intValue() };
> }
> @Override
> public boolean moveNext() {
> return cnt.decrementAndGet() > 0;
> }
> @Override
> public void reset() {
> }
> @Override
>
[jira] [Commented] (CALCITE-3364) Can't group table function result due to a type cast error if table function returns a row with a single value
[
https://issues.apache.org/jira/browse/CALCITE-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935767#comment-16935767
]
Danny Chan commented on CALCITE-3364:
-
Thanks [~anha] for reporting this, just curious, if we only have one column in
the row, why returns an array Object[], can we return an object directly ?
> Can't group table function result due to a type cast error if table function
> returns a row with a single value
> --
>
> Key: CALCITE-3364
> URL: https://issues.apache.org/jira/browse/CALCITE-3364
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Anton Haidai
>Priority: Major
>
> I was not able to find a suitable test, so I'll describe the issue using a
> custom minimal table function returning a row with a single value as an
> example. I believe, that it should be reproducible against any table function.
> There is a simple table function my_dummy() that just prints numbers 1 to 5.
> Simple select works as expected:
> {code}
> SQL:
> select val from table(my_dummy())
> Result:
> val : INTEGER
> 5
> 4
> 3
> 2
> 1
> {code}
> However, when trying to make an aggregation on top of it, there is a class
> cast error:
> {code}
> SQL:
> select val from table(my_dummy())
> group by val
> Error:
> java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to
> java.lang.Integer
> at
> org.apache.calcite.avatica.util.AbstractCursor$IntAccessor.getInt(AbstractCursor.java:541)
> at org.apache.calcite.avatica.AvaticaSite.get(AvaticaSite.java:340)
> at
> org.apache.calcite.avatica.AvaticaResultSet.getObject(AvaticaResultSet.java:393)
> {code}
> Actually, this array Object[] contains a single integer produced by the table
> function. Also I faced similar class cast errors in a generated code making
> hashJoin (in a complex production code), I believe, the cause should be the
> same as in this minimal example.
> Here is this my_dummy() table function implementation (single file):
> {code}
> package com.myapp.tf;
> import org.apache.calcite.DataContext;
> import org.apache.calcite.linq4j.AbstractEnumerable;
> import org.apache.calcite.linq4j.Enumerable;
> import org.apache.calcite.linq4j.Enumerator;
> import org.apache.calcite.linq4j.tree.Types;
> import org.apache.calcite.rel.type.RelDataType;
> import org.apache.calcite.rel.type.RelDataTypeFactory;
> import org.apache.calcite.schema.ScannableTable;
> import org.apache.calcite.schema.TableFunction;
> import org.apache.calcite.schema.impl.AbstractTable;
> import org.apache.calcite.schema.impl.TableFunctionImpl;
> import org.apache.calcite.sql.SqlIdentifier;
> import org.apache.calcite.sql.SqlOperatorBinding;
> import org.apache.calcite.sql.parser.SqlParserPos;
> import org.apache.calcite.sql.type.SqlTypeName;
> import org.apache.calcite.sql.validate.SqlUserDefinedTableFunction;
> import java.util.Collections;
> import java.util.concurrent.atomic.AtomicInteger;
> import static org.apache.calcite.sql.type.OperandTypes.family;
> public class DummyFunction extends SqlUserDefinedTableFunction {
> public static String DUMMY_FUNCTION_NAME = "my_dummy";
> public static final TableFunction DUMMY_TABLE_FUNCTION =
> TableFunctionImpl.create(Types.lookupMethod(
> DummyFunction.class,
> "createDummyTable"
> ));
> public static DummyTable createDummyTable() {
> return new DummyTable();
> }
> public DummyFunction() {
> super(new SqlIdentifier(DUMMY_FUNCTION_NAME, SqlParserPos.ZERO),
> null,
> null,
> family(),
> Collections.emptyList(),
> DUMMY_TABLE_FUNCTION
> );
> }
> @Override
> public RelDataType inferReturnType(SqlOperatorBinding opBinding) {
> RelDataTypeFactory typeFactory = opBinding.getTypeFactory();
> return typeFactory.builder()
> .add("val", SqlTypeName.INTEGER)
> .build();
> }
> }
> class DummyTable extends AbstractTable implements ScannableTable {
> private final AtomicInteger cnt = new AtomicInteger(6);
> @Override
> public Enumerable scan(DataContext root) {
> return new AbstractEnumerable() {
> public Enumerator enumerator() {
> return new Enumerator () {
> @Override
> public Object[] current() {
> return new Object[] { cnt.intValue() };
> }
> @Override
> public boolean moveNext() {
> return cnt.decrementAndGet() > 0;
> }
> @Override
> public void reset() {
> }
>
[jira] [Resolved] (CALCITE-3247) In JDBC adapter, when generating SQL for Hive, transform SUBSTRING function to correct format
[
https://issues.apache.org/jira/browse/CALCITE-3247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3247.
-
Fix Version/s: 1.22.0
Assignee: Danny Chan
Resolution: Fixed
Fixed in
[a7a566b|https://github.com/apache/calcite/commit/a7a566bb004fd73ee6402652f5572ad0012a97c9],
thanks for your PR, [~jackyWoo] !
> In JDBC adapter, when generating SQL for Hive, transform SUBSTRING function
> to correct format
> -
>
> Key: CALCITE-3247
> URL: https://issues.apache.org/jira/browse/CALCITE-3247
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.19.0
>Reporter: Jacky Woo
>Assignee: Danny Chan
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Let's assume sql = SELECT SUBSTRING('ABC', 2)
> When we use HiveSqlDialect and transform "sql", we expect SUBSTRING(' abc',
> 2)，but get SUBSTRING(' abc' FROM 2) which is incorrect sql format in hive.
> So maybe HiveSqlDialect behavior should be changed when transform function
> SUBSTRING:
> # {{SELECT SUBSTRING('ABC', 2) => SELECT SUBSTRING('ABC', 2)}}
> # {{SELECT SUBSTRING('ABC', 2, 3) => SELECT SUBSTRING('ABC', 2, 3) }}
> # {{SELECT SUBSTRING('ABC' FROM 2) => SELECT SUBSTRING('ABC', 2) }}
> # {{SELECT SUBSTRING('ABC' FROM 2 FOR 3) => SELECT SUBSTRING('ABC', 2, 3) }}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3288) linq4j: support Set literals
[ https://issues.apache.org/jira/browse/CALCITE-3288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3288. - Fix Version/s: 1.22.0 Assignee: Danny Chan Resolution: Fixed Fixed in [78c2a00|https://github.com/apache/calcite/commit/78c2a007b5b597f673323b80dde21c23d31f23ea], thanks for your PR, [~xzh_dz] ! > linq4j: support Set literals > > > Key: CALCITE-3288 > URL: https://issues.apache.org/jira/browse/CALCITE-3288 > Project: Calcite > Issue Type: Improvement >Reporter: xzh_dz >Assignee: Danny Chan >Priority: Critical > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-1178) Allow SqlBetweenOperator to compare DATE and TIMESTAMP
[
https://issues.apache.org/jira/browse/CALCITE-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-1178.
-
Fix Version/s: 1.22.0
Resolution: Fixed
Fixed in
[64068d6|https://github.com/apache/calcite/commit/64068d69ceee8450e56c4705d26b7a2f944909b9]
!
> Allow SqlBetweenOperator to compare DATE and TIMESTAMP
> --
>
> Key: CALCITE-1178
> URL: https://issues.apache.org/jira/browse/CALCITE-1178
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Sean Hsuan-Yi Chu
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> An expression such as
> {code}
> date '1999-03-02' between date '1999-03-01' and timestamp '1999-03-03
> 00:00:00.0'
> {code}
> will incur SqlValidatorException since SqlBetweenOperator does not allow DATE
> and TIMESTAMP comparison. In terms of usability, it would be great if this
> type of comparison is allowed in Calcite.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (CALCITE-1178) Allow SqlBetweenOperator to compare DATE and TIMESTAMP
[
https://issues.apache.org/jira/browse/CALCITE-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan reassigned CALCITE-1178:
---
Assignee: Danny Chan (was: Sean Hsuan-Yi Chu)
> Allow SqlBetweenOperator to compare DATE and TIMESTAMP
> --
>
> Key: CALCITE-1178
> URL: https://issues.apache.org/jira/browse/CALCITE-1178
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Sean Hsuan-Yi Chu
>Assignee: Danny Chan
>Priority: Major
>
> An expression such as
> {code}
> date '1999-03-02' between date '1999-03-01' and timestamp '1999-03-03
> 00:00:00.0'
> {code}
> will incur SqlValidatorException since SqlBetweenOperator does not allow DATE
> and TIMESTAMP comparison. In terms of usability, it would be great if this
> type of comparison is allowed in Calcite.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3353) ProjectJoinTransposeRule caused AssertionError when creating a new Join
[
https://issues.apache.org/jira/browse/CALCITE-3353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934134#comment-16934134
]
Danny Chan commented on CALCITE-3353:
-
Thanks [~zabetak], i kind of disagree with your point for 2 reasons:
# The default instance name was changed which would break the compatibility
# There are many instances in one rule which just different with node types or
conventions which i think would make user confused (except that he is very
familiar with Calcite rule planning)
I'm considering if we can
# accurately give out a rule list that don't really need to match the
convention nodes, and change them always match logical nodes
# or if we can figure out a way to parameterized these variables(node types,
the conventions etc.)
I kind of suggest RelFactory to be the only entrance to construct new nodes
during planning, it should support the convention copy and we deprecate the
RelNode#copy method.
> ProjectJoinTransposeRule caused AssertionError when creating a new Join
> ---
>
> Key: CALCITE-3353
> URL: https://issues.apache.org/jira/browse/CALCITE-3353
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Trying to add ProjectJoinTransposeRule to the rule set to optimize sql (using
> VolcanoPlanner).
> Get the following exception：
> Caused by: java.lang.AssertionErrorCaused by: java.lang.AssertionError at
> org.apache.calcite.adapter.enumerable.EnumerableMergeJoin.(EnumerableMergeJoin.java:72)
> at
> org.apache.calcite.adapter.enumerable.EnumerableMergeJoin.copy(EnumerableMergeJoin.java:112)
> at
> org.apache.calcite.adapter.enumerable.EnumerableMergeJoin.copy(EnumerableMergeJoin.java:59)
> at
> org.apache.calcite.rel.rules.ProjectJoinTransposeRule.onMatch(ProjectJoinTransposeRule.java:155)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:703)
> at org.apache.calcite.tools.Programs.lambda$standard$7(Programs.java:466) at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:528) at
> org.apache.calcite.prepare.Prepare.optimize(Prepare.java:196) at
> org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:416) at
> org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:310) at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:830)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:610)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:580)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:247)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:711)
>
> ... 22 more
> This is because the ProjectJoinTransposeRule can match multiple types of Join
> and ProjectJoinTransposeRule creates a new Join using the join.copy() method.
> When the rule applied to EnumerableMergeJoin, and the new Join created by
> this rule may not meet the requirements of EnumerableMergeJoin.
> {code:java}
> EnumerableMergeJoin(
> RelOptCluster cluster,
> RelTraitSet traits,
> RelNode left,
> RelNode right,
> RexNode condition,
> Set variablesSet,
> JoinRelType joinType) {
> super(cluster, traits, left, right, condition, variablesSet, joinType);
> final List collations =
> traits.getTraits(RelCollationTraitDef.INSTANCE);
> assert collations == null || RelCollations.contains(collations,
> joinInfo.leftKeys);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3352) ProjectToWindowRule sets wrong collation on generated Window
[
https://issues.apache.org/jira/browse/CALCITE-3352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934067#comment-16934067
]
Danny Chan commented on CALCITE-3352:
-
Indeed, but the ProjectToWindowRule can match physical nodes, that is the
reason why [~Pavel Gubin] encounter this problem, we should limit this rule to
only match logical nodes because it is a plan rewrite.
> ProjectToWindowRule sets wrong collation on generated Window
>
>
> Key: CALCITE-3352
> URL: https://issues.apache.org/jira/browse/CALCITE-3352
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
>Reporter: Pavel Gubin
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> {{ProjectToWindowRule}} sets the collocation of the {{Project}} it applies to
> on the generated {{LogicalWindow}} which is wrong, because the collation of
> the input can be different and Window doesn't change the collation. So the
> collation of the generated {{LogicalWindow}} should be the same as on the
> input (not the output).
> The following test in {{RelOptRulesTest}} reproduces the issue:
> {code:java}
> @Test public void testWindowOnSortedInput() {
> // Create a customized test with RelCollation trait in the test cluster.
> Tester tester = new TesterImpl(getDiffRepos(), true, true, false, false,
> true, null, null) {
> @Override public RelOptPlanner createPlanner() {
> return new MockRelOptPlanner(Contexts.empty()) {
> @Override public List getRelTraitDefs() {
> return ImmutableList.of(RelCollationTraitDef.INSTANCE);
> }
> @Override public RelTraitSet emptyTraitSet() {
> return RelTraitSet.createEmpty().plus(
> RelCollationTraitDef.INSTANCE.getDefault());
> }
> };
> }
> };
> final HepProgram preProgram = new HepProgramBuilder()
> .addRuleInstance(SortProjectTransposeRule.INSTANCE)
> .build();
> final HepProgram program = HepProgram.builder()
> .addRuleInstance(ProjectToWindowRule.PROJECT)
> .build();
> final String sql = "select mgr, deptno, sum(sal) over (partition by
> deptno)\n"
> + "from emp\n"
> + "order by mgr, deptno";
> RelNode r = checkPlanning(tester, preProgram, new HepPlanner(program),
> sql);
> RelCollation c =
> r.getInput(0).getTraitSet().getTrait(RelCollationTraitDef.INSTANCE);
> assertEquals("Collation is incorrect", "[3, 7]", c.toString());
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3360) SqlValidator throws NPE for unregistered function without implicit type coercion
[ https://issues.apache.org/jira/browse/CALCITE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933953#comment-16933953 ] Danny Chan edited comment on CALCITE-3360 at 9/20/19 4:18 AM: -- Fixed in [883e191|https://github.com/apache/calcite/commit/883e19170a45e0badffa5b8fba04cd536e0f2eb2] ! was (Author: danny0405): Fixed in [df3dc81|https://github.com/apache/calcite/commit/df3dc81f23166c15e72d5414944c69a35c2bd110] ! > SqlValidator throws NPE for unregistered function without implicit type > coercion > - > > Key: CALCITE-3360 > URL: https://issues.apache.org/jira/browse/CALCITE-3360 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.21.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3360) SqlValidator throws NPE for unregistered function without implicit type coercion
[ https://issues.apache.org/jira/browse/CALCITE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan updated CALCITE-3360: Summary: SqlValidator throws NPE for unregistered function without implicit type coercion (was: SqlValidator throws NEP for unregistered function without implicit type coercion ) > SqlValidator throws NPE for unregistered function without implicit type > coercion > - > > Key: CALCITE-3360 > URL: https://issues.apache.org/jira/browse/CALCITE-3360 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.21.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3360) SqlValidator throws NPE for unregistered function without implicit type coercion
[ https://issues.apache.org/jira/browse/CALCITE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934033#comment-16934033 ] Danny Chan commented on CALCITE-3360: - Oh, thanks for correct, [~Chunwei Lei] ! > SqlValidator throws NPE for unregistered function without implicit type > coercion > - > > Key: CALCITE-3360 > URL: https://issues.apache.org/jira/browse/CALCITE-3360 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.21.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3360) SqlValidator throws NEP for unregistered function without implicit type coercion
[ https://issues.apache.org/jira/browse/CALCITE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3360. - Resolution: Fixed Fixed in [df3dc81|https://github.com/apache/calcite/commit/df3dc81f23166c15e72d5414944c69a35c2bd110] ! > SqlValidator throws NEP for unregistered function without implicit type > coercion > - > > Key: CALCITE-3360 > URL: https://issues.apache.org/jira/browse/CALCITE-3360 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.21.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3350) Keep origin type for RexLiteral when deserialized from json string
[
https://issues.apache.org/jira/browse/CALCITE-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933948#comment-16933948
]
Danny Chan edited comment on CALCITE-3350 at 9/20/19 2:16 AM:
--
Personally i'm curious about why the original `STRING` literal is VARCHAR type,
should it still be CHAR(n) ?
was (Author: danny0405):
Personally i'm curious about why the original `STRING` literal is `VARCHAR`
type, should it still be `CHAR(n)` ?
> Keep origin type for RexLiteral when deserialized from json string
> --
>
> Key: CALCITE-3350
> URL: https://issues.apache.org/jira/browse/CALCITE-3350
> Project: Calcite
> Issue Type: Bug
>Reporter: Wang Yanlin
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The json string of sql
> {noformat}
> select ename from emp where job = "abd"
> {noformat}
> is
> {noformat}
> {
> "rels": [
> {
> "id": "0",
> "relOp": "LogicalTableScan",
> "table": [
> "scott",
> "EMP"
> ],
> "inputs": []
> },
> {
> "id": "1",
> "relOp": "LogicalFilter",
> "condition": {
> "op": {
> "name": "=",
> "kind": "EQUALS",
> "syntax": "BINARY"
> },
> "operands": [
> {
> "input": 2,
> "name": "$2"
> },
> {
> "literal": "abd",
> "type": {
> "type": "VARCHAR",
> "nullable": false,
> "precision": 10
> }
> }
> ]
> }
> },
> {
> "id": "2",
> "relOp": "LogicalProject",
> "fields": [
> "ENAME"
> ],
> "exprs": [
> {
> "input": 1,
> "name": "$1"
> }
> ]
> }
> ]
> }
> {noformat}
> The original type of literal "abd" is
> {noformat}
> VARCHAR(10) NOT NULL
> {noformat}
> When deserialized relnode from this json string, the type of literal "abd" is
> changed to
> {noformat}
> CHAR(3) NOT NULL
> {noformat}
> Better to keep the same with original type when deserialized from string.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3350) Keep origin type for RexLiteral when deserialized from json string
[
https://issues.apache.org/jira/browse/CALCITE-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933948#comment-16933948
]
Danny Chan edited comment on CALCITE-3350 at 9/20/19 2:16 AM:
--
Personally i'm curious about why the original `STRING` literal is VARCHAR type,
should it still be CHAR(n) ?
was (Author: danny0405):
Personally i'm curious about why the original `STRING` literal is VARCHAR type,
should it still be CHAR(n) ?
> Keep origin type for RexLiteral when deserialized from json string
> --
>
> Key: CALCITE-3350
> URL: https://issues.apache.org/jira/browse/CALCITE-3350
> Project: Calcite
> Issue Type: Bug
>Reporter: Wang Yanlin
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The json string of sql
> {noformat}
> select ename from emp where job = "abd"
> {noformat}
> is
> {noformat}
> {
> "rels": [
> {
> "id": "0",
> "relOp": "LogicalTableScan",
> "table": [
> "scott",
> "EMP"
> ],
> "inputs": []
> },
> {
> "id": "1",
> "relOp": "LogicalFilter",
> "condition": {
> "op": {
> "name": "=",
> "kind": "EQUALS",
> "syntax": "BINARY"
> },
> "operands": [
> {
> "input": 2,
> "name": "$2"
> },
> {
> "literal": "abd",
> "type": {
> "type": "VARCHAR",
> "nullable": false,
> "precision": 10
> }
> }
> ]
> }
> },
> {
> "id": "2",
> "relOp": "LogicalProject",
> "fields": [
> "ENAME"
> ],
> "exprs": [
> {
> "input": 1,
> "name": "$1"
> }
> ]
> }
> ]
> }
> {noformat}
> The original type of literal "abd" is
> {noformat}
> VARCHAR(10) NOT NULL
> {noformat}
> When deserialized relnode from this json string, the type of literal "abd" is
> changed to
> {noformat}
> CHAR(3) NOT NULL
> {noformat}
> Better to keep the same with original type when deserialized from string.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3350) Keep origin type for RexLiteral when deserialized from json string
[
https://issues.apache.org/jira/browse/CALCITE-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933948#comment-16933948
]
Danny Chan edited comment on CALCITE-3350 at 9/20/19 2:16 AM:
--
Personally i'm curious about why the original `STRING` literal is VARCHAR type,
should it still be CHAR (n) ?
was (Author: danny0405):
Personally i'm curious about why the original `STRING` literal is VARCHAR type,
should it still be CHAR(n) ?
> Keep origin type for RexLiteral when deserialized from json string
> --
>
> Key: CALCITE-3350
> URL: https://issues.apache.org/jira/browse/CALCITE-3350
> Project: Calcite
> Issue Type: Bug
>Reporter: Wang Yanlin
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The json string of sql
> {noformat}
> select ename from emp where job = "abd"
> {noformat}
> is
> {noformat}
> {
> "rels": [
> {
> "id": "0",
> "relOp": "LogicalTableScan",
> "table": [
> "scott",
> "EMP"
> ],
> "inputs": []
> },
> {
> "id": "1",
> "relOp": "LogicalFilter",
> "condition": {
> "op": {
> "name": "=",
> "kind": "EQUALS",
> "syntax": "BINARY"
> },
> "operands": [
> {
> "input": 2,
> "name": "$2"
> },
> {
> "literal": "abd",
> "type": {
> "type": "VARCHAR",
> "nullable": false,
> "precision": 10
> }
> }
> ]
> }
> },
> {
> "id": "2",
> "relOp": "LogicalProject",
> "fields": [
> "ENAME"
> ],
> "exprs": [
> {
> "input": 1,
> "name": "$1"
> }
> ]
> }
> ]
> }
> {noformat}
> The original type of literal "abd" is
> {noformat}
> VARCHAR(10) NOT NULL
> {noformat}
> When deserialized relnode from this json string, the type of literal "abd" is
> changed to
> {noformat}
> CHAR(3) NOT NULL
> {noformat}
> Better to keep the same with original type when deserialized from string.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3350) Keep origin type for RexLiteral when deserialized from json string
[
https://issues.apache.org/jira/browse/CALCITE-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933948#comment-16933948
]
Danny Chan commented on CALCITE-3350:
-
Personally i'm curious about why the original `STRING` literal is `VARCHAR`
type, should it still be `CHAR(n)` ?
> Keep origin type for RexLiteral when deserialized from json string
> --
>
> Key: CALCITE-3350
> URL: https://issues.apache.org/jira/browse/CALCITE-3350
> Project: Calcite
> Issue Type: Bug
>Reporter: Wang Yanlin
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The json string of sql
> {noformat}
> select ename from emp where job = "abd"
> {noformat}
> is
> {noformat}
> {
> "rels": [
> {
> "id": "0",
> "relOp": "LogicalTableScan",
> "table": [
> "scott",
> "EMP"
> ],
> "inputs": []
> },
> {
> "id": "1",
> "relOp": "LogicalFilter",
> "condition": {
> "op": {
> "name": "=",
> "kind": "EQUALS",
> "syntax": "BINARY"
> },
> "operands": [
> {
> "input": 2,
> "name": "$2"
> },
> {
> "literal": "abd",
> "type": {
> "type": "VARCHAR",
> "nullable": false,
> "precision": 10
> }
> }
> ]
> }
> },
> {
> "id": "2",
> "relOp": "LogicalProject",
> "fields": [
> "ENAME"
> ],
> "exprs": [
> {
> "input": 1,
> "name": "$1"
> }
> ]
> }
> ]
> }
> {noformat}
> The original type of literal "abd" is
> {noformat}
> VARCHAR(10) NOT NULL
> {noformat}
> When deserialized relnode from this json string, the type of literal "abd" is
> changed to
> {noformat}
> CHAR(3) NOT NULL
> {noformat}
> Better to keep the same with original type when deserialized from string.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3349) Add Function DDL into SqlKind DDL enum
[ https://issues.apache.org/jira/browse/CALCITE-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933213#comment-16933213 ] Danny Chan commented on CALCITE-3349: - Personally, i'm okey with this change, but i have no strong objections for [~suez1224]'s suggesion, [~ZhenqiuHuang], you decide if to merge these 2 PRs. > Add Function DDL into SqlKind DDL enum > -- > > Key: CALCITE-3349 > URL: https://issues.apache.org/jira/browse/CALCITE-3349 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Zhenqiu Huang >Priority: Minor > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Currently, Create Function, Drop Function are not added into SqlKind DDL > enum. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3354) RelToSqlConverter does not support LogicalTableFunctionScan
[
https://issues.apache.org/jira/browse/CALCITE-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933211#comment-16933211
]
Danny Chan commented on CALCITE-3354:
-
Thanks for reporting [~anha], and welcome to contribute ~
> RelToSqlConverter does not support LogicalTableFunctionScan
> ---
>
> Key: CALCITE-3354
> URL: https://issues.apache.org/jira/browse/CALCITE-3354
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Anton Haidai
>Priority: Minor
>
> Here is the test in *RelBuilderTest* to reproduce the issue using
> *RelBuilder* (the right place for this test is RelToSqlConverterTest, but it
> does not have table functions configured):
> {code}
> @Test
> public void testFailToConvertLogicalTableFunctionScanToSql() {
> final RelBuilder builder = RelBuilder.create(config().build());
> final SqlOperator rampFunction = new MockSqlOperatorTable.RampFunction();
> RelNode root = builder.functionScan(rampFunction, 0, builder.literal(3))
> .build();
> toSql(root, CalciteSqlDialect.DEFAULT);
> }
> private static String toSql(RelNode root, SqlDialect dialect) {
> final RelToSqlConverter converter = new RelToSqlConverter(dialect);
> final SqlNode sqlNode = converter.visitChild(0, root).asStatement();
> return sqlNode.toSqlString(dialect).getSql();
> }
> {code}
> Here is the same issue reproduced using *SQL* (placed in
> *SqlToRelConverterTest*: once again, the test file was selected because of
> table functions infrastructure availability):
> {code}
> @Test
> public void testFailToConvertLogicalTableFunctionScanToSql() {
> String sql = "select * from table(ramp(3))";
> final RelNode rel = tester.convertSqlToRel(sql).rel;
> toSql(rel, CalciteSqlDialect.DEFAULT);
> }
> private static String toSql(RelNode root, SqlDialect dialect) {
> final RelToSqlConverter converter = new RelToSqlConverter(dialect);
> final SqlNode sqlNode = converter.visitChild(0, root).asStatement();
> return sqlNode.toSqlString(dialect).getSql();
> }
> {code}
> Result:
> {code}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.rel.rel2sql.SqlImplementor$Result
> org.apache.calcite.rel.rel2sql.RelToSqlConverter.visit(org.apache.calcite.rel.RelNode)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverter.dispatch(RelToSqlConverter.java:122)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverter.visitChild(RelToSqlConverter.java:128)
> at org.apache.calcite.test.RelBuilderTest.toSql(RelBuilderTest.java:331)
> at
> org.apache.calcite.test.RelBuilderTest.testFailToConvertLogicalTableFunctionScanToSql(RelBuilderTest.java:326)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
[jira] [Commented] (CALCITE-1178) Allow SqlBetweenOperator to compare DATE and TIMESTAMP
[
https://issues.apache.org/jira/browse/CALCITE-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933207#comment-16933207
]
Danny Chan commented on CALCITE-1178:
-
[~jacobroldan], yeah, you are right, 1.21 supports implicit type coercion for
DATE and TIMESTAMP in binary comparison, i think we can support between either.
For current coercion rules, for "date > timestamp", we coerce the DATE to
TIMESTAMP.
[~julianhyde] I have checked MYSQL 5.6 and PostgreSQL 9.6, they have the same
behavior with Calcite.
> Allow SqlBetweenOperator to compare DATE and TIMESTAMP
> --
>
> Key: CALCITE-1178
> URL: https://issues.apache.org/jira/browse/CALCITE-1178
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Sean Hsuan-Yi Chu
>Assignee: Sean Hsuan-Yi Chu
>Priority: Major
>
> An expression such as
> {code}
> date '1999-03-02' between date '1999-03-01' and timestamp '1999-03-03
> 00:00:00.0'
> {code}
> will incur SqlValidatorException since SqlBetweenOperator does not allow DATE
> and TIMESTAMP comparison. In terms of usability, it would be great if this
> type of comparison is allowed in Calcite.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3349) Add Function DDL into SqlKind DDL enum
[ https://issues.apache.org/jira/browse/CALCITE-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933197#comment-16933197 ] Danny Chan commented on CALCITE-3349: - [~walterddr], there is no reason, it is just not implemented yet. > Add Function DDL into SqlKind DDL enum > -- > > Key: CALCITE-3349 > URL: https://issues.apache.org/jira/browse/CALCITE-3349 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Zhenqiu Huang >Priority: Minor > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Currently, Create Function, Drop Function are not added into SqlKind DDL > enum. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3355) Deduce whether CASE and COALESCE may produce NULL values
[ https://issues.apache.org/jira/browse/CALCITE-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933046#comment-16933046 ] Danny Chan commented on CALCITE-3355: - Thanks [~kirillkozlov] for reporting this ~ > Deduce whether CASE and COALESCE may produce NULL values > > > Key: CALCITE-3355 > URL: https://issues.apache.org/jira/browse/CALCITE-3355 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Kirill Kozlov >Priority: Major > > When executing queries like: > "SELECT COALESCE(name, 'unknown') as name_out FROM PCOLLECTION" > (input 'name' is nullable) > There is a need to know whether the result is nullable or not (in this case - > not). During validation stage "COALESCE" is being transformed (via > SqlValidatorImpl.performUnconditionalRewrites) into a "CASE" statement, which > currently does not determine nullability of a result. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3360) SqlValidator throws NEP for unregistered function without implicit type coercion
Danny Chan created CALCITE-3360: --- Summary: SqlValidator throws NEP for unregistered function without implicit type coercion Key: CALCITE-3360 URL: https://issues.apache.org/jira/browse/CALCITE-3360 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.21.0 Reporter: Danny Chan Assignee: Danny Chan Fix For: 1.22.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-2707) Information about distinct aggregation is lost in MATCH_RECOGNIZE
[
https://issues.apache.org/jira/browse/CALCITE-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932181#comment-16932181
]
Danny Chan commented on CALCITE-2707:
-
When can we fix this issue, there is a copy file of Calcite SqlValidatorImpl in
flink-table-planner, and there is no change to patch other Calcite bug fix in
the validator, we should solve this issue as soon as possible and remove the
file SqlValidatorImpl.java.
> Information about distinct aggregation is lost in MATCH_RECOGNIZE
> -
>
> Key: CALCITE-2707
> URL: https://issues.apache.org/jira/browse/CALCITE-2707
> Project: Calcite
> Issue Type: Bug
>Reporter: Dawid Wysakowicz
>Priority: Major
> Labels: match
>
> There is no information that the COUNT aggregation is distinct in a plan for
> a query like e.g. this:
> {code}
> SELECT *
> FROM MyTable
> MATCH_RECOGNIZE (
> MEASURES
>
> AFTER MATCH SKIP PAST LAST ROW
> PATTERN (A B+ C)
> DEFINE
> B AS COUNT(DISTINCT B.id) < 4
> ) AS T
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3353) ProjectJoinTransposeRule caused AssertionError when creating a new Join
[
https://issues.apache.org/jira/browse/CALCITE-3353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932081#comment-16932081
]
Danny Chan commented on CALCITE-3353:
-
Maybe we should forbidden to match convention(non-logical) nodes for these
promotion rules.
> ProjectJoinTransposeRule caused AssertionError when creating a new Join
> ---
>
> Key: CALCITE-3353
> URL: https://issues.apache.org/jira/browse/CALCITE-3353
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
>
> Trying to add ProjectJoinTransposeRule to the rule set to optimize sql (using
> VolcanoPlanner).
> Get the following exception：
> Caused by: java.lang.AssertionErrorCaused by: java.lang.AssertionError at
> org.apache.calcite.adapter.enumerable.EnumerableMergeJoin.(EnumerableMergeJoin.java:72)
> at
> org.apache.calcite.adapter.enumerable.EnumerableMergeJoin.copy(EnumerableMergeJoin.java:112)
> at
> org.apache.calcite.adapter.enumerable.EnumerableMergeJoin.copy(EnumerableMergeJoin.java:59)
> at
> org.apache.calcite.rel.rules.ProjectJoinTransposeRule.onMatch(ProjectJoinTransposeRule.java:155)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:703)
> at org.apache.calcite.tools.Programs.lambda$standard$7(Programs.java:466) at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:528) at
> org.apache.calcite.prepare.Prepare.optimize(Prepare.java:196) at
> org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:416) at
> org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:310) at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:830)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:610)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:580)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:247)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:711)
>
> ... 22 more
> This is because the ProjectJoinTransposeRule can match multiple types of Join
> and ProjectJoinTransposeRule creates a new Join using the join.copy() method.
> When the rule applied to EnumerableMergeJoin, and the new Join created by
> this rule may not meet the requirements of EnumerableMergeJoin.
> {code:java}
> EnumerableMergeJoin(
> RelOptCluster cluster,
> RelTraitSet traits,
> RelNode left,
> RelNode right,
> RexNode condition,
> Set variablesSet,
> JoinRelType joinType) {
> super(cluster, traits, left, right, condition, variablesSet, joinType);
> final List collations =
> traits.getTraits(RelCollationTraitDef.INSTANCE);
> assert collations == null || RelCollations.contains(collations,
> joinInfo.leftKeys);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3351) calcite mysql utf8 throws exception in Chinese
[
https://issues.apache.org/jira/browse/CALCITE-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931072#comment-16931072
]
Danny Chan commented on CALCITE-3351:
-
The exception is thrown from avatica, and it seems that it is related with
CALCITE-2704, there is already a fix from [~vlsi][1].
[1] [https://github.com/apache/calcite-avatica/pull/85]
> calcite mysql utf8 throws exception in Chinese
> --
>
> Key: CALCITE-3351
> URL: https://issues.apache.org/jira/browse/CALCITE-3351
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.19.0
> Environment: Happened on both Windows and MacOS.
>Reporter: cui
>Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Exception was thrown when connecting with mySQL to query Chinese. The
> following SQL was executed:
> select * from \"test\".\"score_new\" where \"name\"= \'催\' limit 1
> ，if change "getDefaultCharset" in "RelDataTypeFactoryImpl" to
> return Charset.forName("UTF8");
> there will be also exception thrown as the following:
> java.sql.SQLException: Error while executing SQL "select * from
> "test"."score_new" where "name"= '催' limit 1": While executing SQL [SELECT
> *java.sql.SQLException: Error while executing SQL "select * from
> "test"."score_new" where "name"= '催' limit 1": While executing SQL [SELECT
> *FROM `score_new`WHERE `name` = u&'\50ac'LIMIT 1] on JDBC sub-schema at
> org.apache.calcite.avatica.Helper.createException(Helper.java:56) at
> org.apache.calcite.avatica.Helper.createException(Helper.java:41) at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
> at QueryMysql.main(QueryMysql.java:42)Caused by: java.lang.RuntimeException:
> While executing SQL [SELECT *FROM `score_new`WHERE `name` = u&'\50ac'
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (CALCITE-3282) HiveSqlDialect unparse Interger type as Int in order to be compatible with Hive1.x
[ https://issues.apache.org/jira/browse/CALCITE-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3282. - Fix Version/s: 1.22.0 Assignee: Danny Chan Resolution: Fixed Fixed in [148bfd3|https://github.com/apache/calcite/commit/148bfd329413c0272395cc0b7c322b3c5a34b667], thanks for your PR, [~ffmax] ! > HiveSqlDialect unparse Interger type as Int in order to be compatible with > Hive1.x > -- > > Key: CALCITE-3282 > URL: https://issues.apache.org/jira/browse/CALCITE-3282 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: feng huang >Assignee: Danny Chan >Priority: Minor > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > > Every database might have different type or same type but different type > name, therefore making every SqlDialect unparse their own data type is a > suitable way. > For example, there is a sql “select cast(col as int) from table” change to > hive sql "select cast(col as integer) from table", but "integer" is not > allowed in hive. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (CALCITE-3282) HiveSqlDialect unparse Interger type as Int in order to be compatible with Hive1.x
[ https://issues.apache.org/jira/browse/CALCITE-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan updated CALCITE-3282: Summary: HiveSqlDialect unparse Interger type as Int in order to be compatible with Hive1.x (was: Make every SqlDialect unparse their own data type) > HiveSqlDialect unparse Interger type as Int in order to be compatible with > Hive1.x > -- > > Key: CALCITE-3282 > URL: https://issues.apache.org/jira/browse/CALCITE-3282 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: feng huang >Priority: Minor > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > Every database might have different type or same type but different type > name, therefore making every SqlDialect unparse their own data type is a > suitable way. > For example, there is a sql “select cast(col as int) from table” change to > hive sql "select cast(col as integer) from table", but "integer" is not > allowed in hive. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-2772) Support varargs for user-defined functions (UDFs)
[
https://issues.apache.org/jira/browse/CALCITE-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930393#comment-16930393
]
Danny Chan commented on CALCITE-2772:
-
The change is breaking, because the user defined function are user api, we
should be cautious if we really need the arguments to be variable.
> Support varargs for user-defined functions (UDFs)
> -
>
> Key: CALCITE-2772
> URL: https://issues.apache.org/jira/browse/CALCITE-2772
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Major
> Labels: pull-request-available
> Attachments: support_varargs_udf.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Support varargs for user-defined functions as the case followed:
> {code:java}
> public class ConcatWs {
> public String eval(String sep, String... strs) {...}
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-2772) Support varargs for user-defined functions (UDFs)
[
https://issues.apache.org/jira/browse/CALCITE-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930390#comment-16930390
]
Danny Chan commented on CALCITE-2772:
-
Let me repeat the review comments again:
Before the review start, can you share your cases why we need to support
variable arguments for UDF/UDAF/UDTF ? We have already declare multiple number
of arguments when registering the functions into the catalog.
For example, we can declare both my_udf(string, string) and my_udf(string,
string, string), so do we really need to support freely variable length
argument functions ?
> Support varargs for user-defined functions (UDFs)
> -
>
> Key: CALCITE-2772
> URL: https://issues.apache.org/jira/browse/CALCITE-2772
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Major
> Labels: pull-request-available
> Attachments: support_varargs_udf.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Support varargs for user-defined functions as the case followed:
> {code:java}
> public class ConcatWs {
> public String eval(String sep, String... strs) {...}
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3349) Add Function DDL into SqlKind DDL enum
[ https://issues.apache.org/jira/browse/CALCITE-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930257#comment-16930257 ] Danny Chan commented on CALCITE-3349: - I'm kind of considering if we can add some tests for these sql kind thing, if is very prone to miss some kind classification because all the items are enumerations. > Add Function DDL into SqlKind DDL enum > -- > > Key: CALCITE-3349 > URL: https://issues.apache.org/jira/browse/CALCITE-3349 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Zhenqiu Huang >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Currently, Create Function, Drop Function are not added into SqlKind DDL > enum. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (CALCITE-3331) Support implicit type cast for operators that use single operand family checker
[
https://issues.apache.org/jira/browse/CALCITE-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3331.
-
Resolution: Fixed
Thanks [~zabetak] and fixed in
[d3c7183|https://github.com/apache/calcite/commit/d3c718328d4c83fb24007c8349b31b420187]
!
> Support implicit type cast for operators that use single operand family
> checker
> ---
>
> Key: CALCITE-3331
> URL: https://issues.apache.org/jira/browse/CALCITE-3331
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When the FamilyOperandTypeChecker is used to check single operand data type,
> support implicit type coercion if we can.
> Now some of the sql operator override method #checkOperandTypes, and use the
> SqlSingleOperandTypeChecker#checkSingleOperandType to check the operand data
> type, we should support the implicit type coercion for these operators.
> One impl that need to note:
> we seem always pass the "iformalOperand" as "0" with method
> SqlSingleOperandTypeChecker#checkSingleOperandType when check the single
> operand, we need to pass in the real operand index in the call to the checker.
>
> Final solution:
> Caution that we could not(shouldn't) implement implicit type coercion for
> this checker,
> implicit type coercion has side effect(modify the AST), if this single
> operand checker is
> subsumed in a composite rule(OR or AND), we can not make any side effect if we
> can not make sure that all the single operands type check are passed(with
> type coercion).
> But there is an exception: only if the call has just one operand, for this
> case,
> use \{@link SqlOperandTypeChecker#checkOperandTypes} instead.
> We decide to fix these operators separately.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (CALCITE-3331) Support implicit type cast for operators that use single operand family checker
[
https://issues.apache.org/jira/browse/CALCITE-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3331.
-
Resolution: Fixed
Fixed in
[1a9b87f|https://github.com/apache/calcite/commit/1a9b87f673e45b86df27eb544f0312be0f95b430]
!
> Support implicit type cast for operators that use single operand family
> checker
> ---
>
> Key: CALCITE-3331
> URL: https://issues.apache.org/jira/browse/CALCITE-3331
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When the FamilyOperandTypeChecker is used to check single operand data type,
> support implicit type coercion if we can.
> Now some of the sql operator override method #checkOperandTypes, and use the
> SqlSingleOperandTypeChecker#checkSingleOperandType to check the operand data
> type, we should support the implicit type coercion for these operators.
> One impl that need to note:
> we seem always pass the "iformalOperand" as "0" with method
> SqlSingleOperandTypeChecker#checkSingleOperandType when check the single
> operand, we need to pass in the real operand index in the call to the checker.
>
> Final solution:
> Caution that we could not(shouldn't) implement implicit type coercion for
> this checker,
> implicit type coercion has side effect(modify the AST), if this single
> operand checker is
> subsumed in a composite rule(OR or AND), we can not make any side effect if we
> can not make sure that all the single operands type check are passed(with
> type coercion).
> But there is an exception: only if the call has just one operand, for this
> case,
> use \{@link SqlOperandTypeChecker#checkOperandTypes} instead.
> We decide to fix these operators separately.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3331) Support implicit type cast for operators that use single operand family checker
[
https://issues.apache.org/jira/browse/CALCITE-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3331:
Description:
When the FamilyOperandTypeChecker is used to check single operand data type,
support implicit type coercion if we can.
Now some of the sql operator override method #checkOperandTypes, and use the
SqlSingleOperandTypeChecker#checkSingleOperandType to check the operand data
type, we should support the implicit type coercion for these operators.
One impl that need to note:
we seem always pass the "iformalOperand" as "0" with method
SqlSingleOperandTypeChecker#checkSingleOperandType when check the single
operand, we need to pass in the real operand index in the call to the checker.
Final solution:
Caution that we could not(shouldn't) implement implicit type coercion for this
checker,
implicit type coercion has side effect(modify the AST), if this single operand
checker is
subsumed in a composite rule(OR or AND), we can not make any side effect if we
can not make sure that all the single operands type check are passed(with type
coercion).
But there is an exception: only if the call has just one operand, for this case,
use \{@link SqlOperandTypeChecker#checkOperandTypes} instead.
We decide to fix these operators separately.
was:
When the FamilyOperandTypeChecker is used to check single operand data type,
support implicit type coercion if we can.
Now some of the sql operator override method #checkOperandTypes, and use the
SqlSingleOperandTypeChecker#checkSingleOperandType to check the operand data
type, we should support the implicit type coercion for these operators.
One impl that need to note:
we seem always pass the "iformalOperand" as "0" with method
SqlSingleOperandTypeChecker#checkSingleOperandType when check the single
operand, we need to pass in the real operand index in the call to the checker.
> Support implicit type cast for operators that use single operand family
> checker
> ---
>
> Key: CALCITE-3331
> URL: https://issues.apache.org/jira/browse/CALCITE-3331
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Minor
> Fix For: 1.22.0
>
>
> When the FamilyOperandTypeChecker is used to check single operand data type,
> support implicit type coercion if we can.
> Now some of the sql operator override method #checkOperandTypes, and use the
> SqlSingleOperandTypeChecker#checkSingleOperandType to check the operand data
> type, we should support the implicit type coercion for these operators.
> One impl that need to note:
> we seem always pass the "iformalOperand" as "0" with method
> SqlSingleOperandTypeChecker#checkSingleOperandType when check the single
> operand, we need to pass in the real operand index in the call to the checker.
>
> Final solution:
> Caution that we could not(shouldn't) implement implicit type coercion for
> this checker,
> implicit type coercion has side effect(modify the AST), if this single
> operand checker is
> subsumed in a composite rule(OR or AND), we can not make any side effect if we
> can not make sure that all the single operands type check are passed(with
> type coercion).
> But there is an exception: only if the call has just one operand, for this
> case,
> use \{@link SqlOperandTypeChecker#checkOperandTypes} instead.
> We decide to fix these operators separately.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3331) Support implicit type cast for operators that use single operand family checker
[ https://issues.apache.org/jira/browse/CALCITE-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan updated CALCITE-3331: Summary: Support implicit type cast for operators that use single operand family checker (was: Support implicit type cast for single operand family checker) > Support implicit type cast for operators that use single operand family > checker > --- > > Key: CALCITE-3331 > URL: https://issues.apache.org/jira/browse/CALCITE-3331 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Minor > Fix For: 1.22.0 > > > When the FamilyOperandTypeChecker is used to check single operand data type, > support implicit type coercion if we can. > Now some of the sql operator override method #checkOperandTypes, and use the > SqlSingleOperandTypeChecker#checkSingleOperandType to check the operand data > type, we should support the implicit type coercion for these operators. > One impl that need to note: > we seem always pass the "iformalOperand" as "0" with method > SqlSingleOperandTypeChecker#checkSingleOperandType when check the single > operand, we need to pass in the real operand index in the call to the checker. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (CALCITE-3331) Support implicit type cast for single operand family checker
[ https://issues.apache.org/jira/browse/CALCITE-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan updated CALCITE-3331: Description: When the FamilyOperandTypeChecker is used to check single operand data type, support implicit type coercion if we can. Now some of the sql operator override method #checkOperandTypes, and use the SqlSingleOperandTypeChecker#checkSingleOperandType to check the operand data type, we should support the implicit type coercion for these operators. One impl that need to note: we seem always pass the "iformalOperand" as "0" with method SqlSingleOperandTypeChecker#checkSingleOperandType when check the single operand, we need to pass in the real operand index in the call to the checker. was: When the FamilyOperandTypeChecker is used to check single operand data type, support implicit type coercion if we can. Now some of the sql operator override method #checkOperandTypes, and use the SqlSingleOperandTypeChecker#checkSingleOperandType to check the operand data type, we should support the implicit type coercion for these operators. > Support implicit type cast for single operand family checker > > > Key: CALCITE-3331 > URL: https://issues.apache.org/jira/browse/CALCITE-3331 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Minor > Fix For: 1.22.0 > > > When the FamilyOperandTypeChecker is used to check single operand data type, > support implicit type coercion if we can. > Now some of the sql operator override method #checkOperandTypes, and use the > SqlSingleOperandTypeChecker#checkSingleOperandType to check the operand data > type, we should support the implicit type coercion for these operators. > One impl that need to note: > we seem always pass the "iformalOperand" as "0" with method > SqlSingleOperandTypeChecker#checkSingleOperandType when check the single > operand, we need to pass in the real operand index in the call to the checker. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3331) Support implicit type cast for single operand family checker
Danny Chan created CALCITE-3331: --- Summary: Support implicit type cast for single operand family checker Key: CALCITE-3331 URL: https://issues.apache.org/jira/browse/CALCITE-3331 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.20.0 Reporter: Danny Chan Assignee: Danny Chan Fix For: 1.22.0 When the FamilyOperandTypeChecker is used to check single operand data type, support implicit type coercion if we can. Now some of the sql operator override method #checkOperandTypes, and use the SqlSingleOperandTypeChecker#checkSingleOperandType to check the operand data type, we should support the implicit type coercion for these operators. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-2166) Cumulative cost of RelSubset.best RelNode is increased after calling RelSubset.propagateCostImprovements() for input RelNodes
[
https://issues.apache.org/jira/browse/CALCITE-2166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925043#comment-16925043
]
Danny Chan commented on CALCITE-2166:
-
Thanks [~xndai],
Well, i think there are 2 cases that option 1 & 2 can not fix up the problem
perfectly:
# one rel may belongs to multi RelSubSet, but because we alway use the DP
first search algorithm to update parents of one RelSubSet, the other RelSubSet
may also be affected.
# The plan may be a DAG, one rel may have multiple parents.
I have reviewed your PR, it seems that you choose the #2 proposal, not #1, i
see you add a for loop in each layer of the DP search, can you give a
performance test about how much overhead the patch has introduced ?
Because #1 and #2 are both not perfect fix, we may need to choose #1 while #2
has some overhead but not significant improvement.
PS: i'm still a little skeptical about whether this patch do better or worse,
just like Julian said, with CALCITE-2018, we just comes from a metadata
breaking state to another one.
> Cumulative cost of RelSubset.best RelNode is increased after calling
> RelSubset.propagateCostImprovements() for input RelNodes
> -
>
> Key: CALCITE-2166
> URL: https://issues.apache.org/jira/browse/CALCITE-2166
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.15.0
>Reporter: Volodymyr Vysotskyi
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> After calling {{RelSubset.propagateCostImprovements()}} cumulative cost of
> {{RelSubset.best}} {{RelNode}} may be increased due to the increase of the
> non-cumulative cost caused by changing of input best {{RelNode}}.
> To observe this issue, add this code:
> {code:java}
> if (subset.best != null) {
> RelOptCost bestCost = getCost(subset.best,
> RelMetadataQuery.instance());
> if (!subset.bestCost.equals(bestCost)) {
> throw new AssertionError(
> "relSubset [" + subset.getDescription()
> + "] has wrong best cost "
> + subset.bestCost + ". Correct cost is " + bestCost);
> }
> }
> {code}
> into {{VolcanoPlanner.validate()}} method (line 907).
> List of unit tests which fail with this check:
> {noformat}
> Failed tests:
>
> MaterializationTest.testJoinMaterializationUKFK9:1823->checkMaterialize:198->checkMaterialize:205->checkThatMaterialize:233
> relSubset [rel#226287:Subset#8.ENUMERABLE.[]] has wrong best cost {221.5
> rows, 128.25 cpu, 0.0 io}. Correct cost is {233.0 rows, 178.0 cpu, 0.0 io}
> ScannableTableTest.testPFPushDownProjectFilterAggregateNested:279 relSubset
> [rel#12950:Subset#5.ENUMERABLE.[]] has wrong best cost {63.8 rows, 62.308
> cpu, 0.0 io}. Correct cost is {70.4 rows, 60.404 cpu, 0.0 io}
> ScannableTableTest.testPFTableRefusesFilterCooperative:221 relSubset
> [rel#13382:Subset#2.ENUMERABLE.[]] has wrong best cost {81.0 rows, 181.01
> cpu, 0.0 io}. Correct cost is {150.5 rows, 250.505 cpu, 0.0 io}
> ScannableTableTest.testProjectableFilterableCooperative:148 relSubset
> [rel#13611:Subset#2.ENUMERABLE.[]] has wrong best cost {81.0 rows, 181.01
> cpu, 0.0 io}. Correct cost is {150.5 rows, 250.505 cpu, 0.0 io}
> ScannableTableTest.testProjectableFilterableNonCooperative:165 relSubset
> [rel#13754:Subset#2.ENUMERABLE.[]] has wrong best cost {81.0 rows, 181.01
> cpu, 0.0 io}. Correct cost is {150.5 rows, 250.505 cpu, 0.0 io}
> FrameworksTest.testUpdate:336->executeQuery:367 relSubset
> [rel#22533:Subset#2.ENUMERABLE.any] has wrong best cost {19.5 rows, 37.75
> cpu, 0.0 io}. Correct cost is {22.575 rows, 52.58 cpu, 0.0 io}
> {noformat}
> For the test {{MaterializationTest.testJoinMaterializationUKFK9}} initial
> best plan was:
> {noformat}
> EnumerableProject(empid0=[$5], empid00=[$5], deptno0=[$7]): rowcount = 15.0,
> cumulative cost = {15.0 rows, 45.0 cpu, 0.0 io}, id = 3989
> EnumerableJoin(subset=[rel#3988:Subset#34.ENUMERABLE.[]], condition=[=($1,
> $7)], joinType=[inner]): rowcount = 15.0, cumulative cost = {116.0 rows, 0.0
> cpu, 0.0 io}, id = 4797
> EnumerableFilter(subset=[rel#4274:Subset#47.ENUMERABLE.[0]],
> condition=[=(CAST($2):VARCHAR CHARACTER SET "ISO-8859-1" COLLATE
> "ISO-8859-1$en_US$primary", 'Bill')]): rowcount = 1.0, cumulative cost = {1.0
> rows, 1.0 cpu, 0.0 io}, id = 16522
> EnumerableTableScan(subset=[rel#158:Subset#11.ENUMERABLE.[0]],
> table=[[hr, m0]]): rowcount = 1.0, cumulative cost = {0.0 rows, 1.0 cpu, 0.0
> io}, id = 79
> EnumerableTableScan(subset=[rel#115:Subset#5.ENUMERABLE.[]], table=[[hr,
>
[jira] [Resolved] (CALCITE-3321) Bigquery does not have correct casing rules
[ https://issues.apache.org/jira/browse/CALCITE-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3321. - Fix Version/s: 1.21.0 Assignee: Danny Chan Resolution: Fixed Fixed in [2cdf747|https://github.com/apache/calcite/commit/2cdf7478c86393bb3d7c6d554d0a820f2cf302b9], thanks for your PR, [~lindseycat] ! > Bigquery does not have correct casing rules > --- > > Key: CALCITE-3321 > URL: https://issues.apache.org/jira/browse/CALCITE-3321 > Project: Calcite > Issue Type: Bug >Reporter: Lindsey Meyer >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 40m > Remaining Estimate: 0h > > Bigquery has some strange case sensitivity rules, so I think it should leave > casing unchanged and make it not case sensitive > See rules here: > https://cloud.google.com/bigquery/docs/reference/standard-sql/lexical#case_sensitivity -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-2302:
Description:
Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
Implicit type cast is an useful function for many cases, So we should support
this.
I checkout Calcite code and found that:
# Now we use a validator to validate our operands types[ through kinds of
namespaces and scopes ]
# Most of the validations will finally goes to
{code:java}
SqlOperator.validateOperands
{code}
# which will use validation logic defined in corresponding
SqlOperandTypeChecker
What i'm confused about is where should i put the implicit type cast logic in?
I figured out 2 ways:
# Supply a tool class/rules to add casts into a parsed SqlNode tree which will
then go through the validation logic later on.
# Unleash the validation logic in kinds of SqlOperandTypeChecker, then modify
the RelNode/RexNodes tree converted from a validated SqlNode tree to add in
casts through custom RelOptRules.
So guys, which of the 2 ways should i go, or if there are better way to do this?
I need your help.
Updated 18-05-30:
Hi guys, i have made a PR in
[CALCITE-2302|https://github.com/apache/calcite/pull/706]
This is design doc: [Calcite Implicit Type Cast
Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
This is the type conversion matrix: [Type Conversion
Matrix|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
I really appreciate your suggestions, thx.
was:
Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
Implicit type cast is an useful function for many cases, So we should support
this.
I checkout Calcite code and found that:
# Now we use a validator to validate our operands types[ through kinds of
namespaces and scopes ]
# Most of the validations will finally goes to
{code:java}
SqlOperator.validateOperands
{code}
# which will use validation logic defined in corresponding
SqlOperandTypeChecker
What i'm confused about is where should i put the implicit type cast logic in?
I figured out 2 ways:
# Supply a tool class/rules to add casts into a parsed SqlNode tree which will
then go through the validation logic later on.
# Unleash the validation logic in kinds of SqlOperandTypeChecker, then modify
the RelNode/RexNodes tree converted from a validated SqlNode tree to add in
casts through custom RelOptRules.
So guys, which of the 2 ways should i go, or if there are better way to do this?
I need your help.
Updated 18-05-30:
Hi guys, i have made a PR in
[CALCITE-2302|https://github.com/apache/calcite/pull/706]
This is design doc: [Calcite Implicit Type Cast
Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
This is the conversion types mapping: [Conversion Types
Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
I really appreciate your suggestions, thx.
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
>
[jira] [Resolved] (CALCITE-3310) Approximate and exact aggregate calls are recognized as the same during sql-to-rel conversion
[
https://issues.apache.org/jira/browse/CALCITE-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3310.
-
Resolution: Fixed
Fixed in
[81c768e|https://github.com/apache/calcite/commit/81c768e60078bcb6c8c78057d80090d27ac86030]
!
> Approximate and exact aggregate calls are recognized as the same during
> sql-to-rel conversion
> -
>
> Key: CALCITE-3310
> URL: https://issues.apache.org/jira/browse/CALCITE-3310
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> For sql:
> {code:sql}
> SELECT empno, count(distinct ename)
> approx_count_distinct(ename)
> FROM emp
> GROUP BY empno
> {code}
> After sql-to-rel conversion, the plan is:
> {code:sql}
> LogicalProject(EMPNO=[$0], EXPR$1=[$1], EXPR$2=[$1])
> LogicalAggregate(group=[{0}], EXPR$1=[COUNT(DISTINCT $1)])
> LogicalProject(EMPNO=[$0], ENAME=[$1])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (CALCITE-3315) Multiple failures in Druid IT tests due to implicit casts
[ https://issues.apache.org/jira/browse/CALCITE-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920015#comment-16920015 ] Danny Chan edited comment on CALCITE-3315 at 8/31/19 4:25 AM: -- [~zabetak] I'm so sorry to be a blocker. I have tried many times to install the calcite-test-dataset but failed, many times because of the files downloading. If i do mvn install skipping tests, it succeed, but i still can not connect to the Druid. I tried to fire the PR first to fix the plan change(expected). Can you please help to make sure the tests result are still correct ? was (Author: danny0405): [~zabetak] I'm so sorry to be a blocker, but i have tried many times to install the calcite-test-dataset but failed, many times because of the files downloading. If i install skipping tests, if succeed, but i still can not connect to the Druid. I tried to fire the PR first to fix the plan change(expected). Can you please help to make sure the tests result are still correct ? > Multiple failures in Druid IT tests due to implicit casts > -- > > Key: CALCITE-3315 > URL: https://issues.apache.org/jira/browse/CALCITE-3315 > Project: Calcite > Issue Type: Bug >Reporter: Stamatis Zampetakis >Priority: Blocker > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 10m > Remaining Estimate: 0h > > After merging CALCITE-2302 there are 12 test failures in DruidAdapterIT and > DruidAdapter2IT. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-3315) Multiple failures in Druid IT tests due to implicit casts
[ https://issues.apache.org/jira/browse/CALCITE-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920015#comment-16920015 ] Danny Chan commented on CALCITE-3315: - [~zabetak] I'm so sorry to be a blocker, but i have tried many times to install the calcite-test-dataset but failed, many times because of the files downloading. If i install skipping tests, if succeed, but i still can not connect to the Druid. I tried to fire the PR first to fix the plan change(expected). Can you please help to make sure the tests result are still correct ? > Multiple failures in Druid IT tests due to implicit casts > -- > > Key: CALCITE-3315 > URL: https://issues.apache.org/jira/browse/CALCITE-3315 > Project: Calcite > Issue Type: Bug >Reporter: Stamatis Zampetakis >Priority: Blocker > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 10m > Remaining Estimate: 0h > > After merging CALCITE-2302 there are 12 test failures in DruidAdapterIT and > DruidAdapter2IT. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (CALCITE-3311) Add doc to site for implicit type coercion
[ https://issues.apache.org/jira/browse/CALCITE-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3311. - Resolution: Fixed Fixed in [72680df|https://github.com/apache/calcite/commit/72680df93ef089abd0a7c0ac8cdc6253619c2ebe] ! > Add doc to site for implicit type coercion > -- > > Key: CALCITE-3311 > URL: https://issues.apache.org/jira/browse/CALCITE-3311 > Project: Calcite > Issue Type: Sub-task > Components: site >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-3315) Multiple failures in Druid IT tests due to implicit casts
[ https://issues.apache.org/jira/browse/CALCITE-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919626#comment-16919626 ] Danny Chan commented on CALCITE-3315: - Thanks [~zabetak] for reporting, fixing ~ > Multiple failures in Druid IT tests due to implicit casts > -- > > Key: CALCITE-3315 > URL: https://issues.apache.org/jira/browse/CALCITE-3315 > Project: Calcite > Issue Type: Bug >Reporter: Stamatis Zampetakis >Priority: Blocker > Fix For: 1.21.0 > > > After merging CALCITE-2302 there are 12 test failures in DruidAdapterIT and > DruidAdapter2IT. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-3312) TIMESTAMPDIFF cannot be converted to SQL
[
https://issues.apache.org/jira/browse/CALCITE-3312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919517#comment-16919517
]
Danny Chan commented on CALCITE-3312:
-
Thanks [~Pavel Gubin] for reporting this, maybe we can overwrite the
unparseCall logic to handle this special function.
> TIMESTAMPDIFF cannot be converted to SQL
>
>
> Key: CALCITE-3312
> URL: https://issues.apache.org/jira/browse/CALCITE-3312
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Pavel Gubin
>Priority: Major
>
> The following test in {{RelToSqlConverterTest}}:
> {code}
> @Test public void testTimestampDiff() {
> String query = "SELECT {fn TIMESTAMPDIFF(SQL_TSI_YEAR,TIMESTAMP
> '2016-01-01 00:00:00', TIMESTAMP '2017-01-01 00:00:00')}";
> String expected = "";
> sql(query).ok(expected);
> }
> {code}
> fails with:
> {noformat}
> java.lang.UnsupportedOperationException: class
> org.apache.calcite.sql.SqlSyntax$6: SPECIAL
> at org.apache.calcite.util.Util.needToImplement(Util.java:967)
> at org.apache.calcite.sql.SqlSyntax$6.unparse(SqlSyntax.java:116)
> at org.apache.calcite.sql.SqlOperator.unparse(SqlOperator.java:348)
> at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
> at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:105)
> at org.apache.calcite.sql.SqlUtil.unparseBinarySyntax(SqlUtil.java:382)
> at org.apache.calcite.sql.SqlSyntax$3.unparse(SqlSyntax.java:65)
> at org.apache.calcite.sql.SqlOperator.unparse(SqlOperator.java:348)
> at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
> at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:108)
> at
> org.apache.calcite.sql.fun.SqlCastFunction.unparse(SqlCastFunction.java:174)
> at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
> at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:108)
> at org.apache.calcite.sql.SqlNodeList.commaList(SqlNodeList.java:121)
> at
> org.apache.calcite.sql.SqlOperator.unparseListClause(SqlOperator.java:365)
> at
> org.apache.calcite.sql.SqlOperator.unparseListClause(SqlOperator.java:354)
> at
> org.apache.calcite.sql.SqlSelectOperator.unparse(SqlSelectOperator.java:152)
> at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
> at org.apache.calcite.sql.SqlSelect.unparse(SqlSelect.java:230)
> at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:152)
> at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:157)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.toSql(RelToSqlConverterTest.java:145)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.access$100(RelToSqlConverterTest.java:77)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.exec(RelToSqlConverterTest.java:3634)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.ok(RelToSqlConverterTest.java:3609)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.testTimestampDiff(RelToSqlConverterTest.java:1942)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
>
[jira] [Commented] (CALCITE-3310) Approximate and exact aggregate calls are recognized as the same during sql-to-rel conversion
[
https://issues.apache.org/jira/browse/CALCITE-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919513#comment-16919513
]
Danny Chan commented on CALCITE-3310:
-
The plan is wrong and the agg output should not be reused.
> Approximate and exact aggregate calls are recognized as the same during
> sql-to-rel conversion
> -
>
> Key: CALCITE-3310
> URL: https://issues.apache.org/jira/browse/CALCITE-3310
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.21.0
>
>
> For sql:
> {code:sql}
> SELECT empno, count(distinct ename)
> approx_count_distinct(ename)
> FROM emp
> GROUP BY empno
> {code}
> After sql-to-rel conversion, the plan is:
> {code:sql}
> LogicalProject(EMPNO=[$0], EXPR$1=[$1], EXPR$2=[$1])
> LogicalAggregate(group=[{0}], EXPR$1=[COUNT(DISTINCT $1)])
> LogicalProject(EMPNO=[$0], ENAME=[$1])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3282) Make every SqlDialect unparse their own data type
[ https://issues.apache.org/jira/browse/CALCITE-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919451#comment-16919451 ] Danny Chan commented on CALCITE-3282: - Let's just ignore unparsing the DDL first, Calcite's DDL has much difference for different sql engines, especially Hive, i think it needs much more effort if we want to unparse the DDL based on sql dialects. > Make every SqlDialect unparse their own data type > - > > Key: CALCITE-3282 > URL: https://issues.apache.org/jira/browse/CALCITE-3282 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: feng huang >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Every database might have different type or same type but different type > name, therefore making every SqlDialect unparse their own data type is a > suitable way. > For example, there is a sql “select cast(col as int) from table” change to > hive sql "select cast(col as integer) from table", but "integer" is not > allowed in hive. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3311) Add doc to site for implicit type coercion
Danny Chan created CALCITE-3311: --- Summary: Add doc to site for implicit type coercion Key: CALCITE-3311 URL: https://issues.apache.org/jira/browse/CALCITE-3311 Project: Calcite Issue Type: Sub-task Components: site Affects Versions: 1.20.0 Reporter: Danny Chan Assignee: Danny Chan Fix For: 1.21.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-3307) PigRelExTest fails on Windows
[
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919296#comment-16919296
]
Danny Chan commented on CALCITE-3307:
-
Thanks for the confirm, Ruben !
> PigRelExTest fails on Windows
> -
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the
> Hadoop binaries.
> at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
> at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
> at org.apache.hadoop.util.Shell.(Shell.java:390)
> at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
> at
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
> at
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
> at
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
> at
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
> at
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
> at
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
> at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
> at org.apache.pig.PigServer.(PigServer.java:231)
> at org.apache.pig.PigServer.(PigServer.java:219)
> at org.apache.pig.PigServer.(PigServer.java:211)
> at org.apache.pig.PigServer.(PigServer.java:207)
> at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
> at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
> at
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> {noformat}
> This issue has arisen many times; see HADOOP-10775 and SPARK-2356.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (CALCITE-3310) Approximate and exact aggregate calls are recognized as the same during sql-to-rel conversion
Danny Chan created CALCITE-3310:
---
Summary: Approximate and exact aggregate calls are recognized as
the same during sql-to-rel conversion
Key: CALCITE-3310
URL: https://issues.apache.org/jira/browse/CALCITE-3310
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.20.0
Reporter: Danny Chan
Assignee: Danny Chan
Fix For: 1.21.0
For sql:
{code:sql}
SELECT empno, count(distinct ename)
approx_count_distinct(ename)
FROM emp
GROUP BY empno
{code}
After sql-to-rel conversion, the plan is:
{code:sql}
LogicalProject(EMPNO=[$0], EXPR$1=[$1], EXPR$2=[$1])
LogicalAggregate(group=[{0}], EXPR$1=[COUNT(DISTINCT $1)])
LogicalProject(EMPNO=[$0], ENAME=[$1])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (CALCITE-3269) Returns integer for VARCHAR and INT division of PostgreSql dialect
[ https://issues.apache.org/jira/browse/CALCITE-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3269. - Resolution: Invalid As discussed in CALCITE-2302, we have chose the PostgreSQL style for binary arithmetic with string. If user want to switch to the MySQL style, they can make a customized TypeCoercion component. > Returns integer for VARCHAR and INT division of PostgreSql dialect > -- > > Key: CALCITE-3269 > URL: https://issues.apache.org/jira/browse/CALCITE-3269 > Project: Calcite > Issue Type: Sub-task > Components: core >Affects Versions: 1.21.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > > We are planning to fix this for PostgreSQL, but maybe not in 1.21.0. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-3122) Convert Pig Latin scripts into Calcite logical plans
[ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918610#comment-16918610 ] Danny Chan commented on CALCITE-3122: - [~khaitran] Do you mean the Kerb Auth or something ? > Convert Pig Latin scripts into Calcite logical plans > - > > Key: CALCITE-3122 > URL: https://issues.apache.org/jira/browse/CALCITE-3122 > Project: Calcite > Issue Type: New Feature > Components: core, piglet >Reporter: Khai Tran >Assignee: Julian Hyde >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > We create an internal Calcite repo at LinkedIn and develop APIs to parse any > Pig Latin scripts into Calcite logical plan. The code was tested in nearly > ~1000 Pig scripts written at LinkedIn. > Changes: > 1. piglet: main conversion code live there, include: > * APIs to convert any Pig scripts into RelNode plans or SQL statements > * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan > (DAGs) > * Convert Pig schemas into RelDatatype > * Traverse through Pig expression plan and convert Pig expressions into > RexNodes > * Map some basic Pig UDFs to Calcite SQL operators > * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both > Java and Python > * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes > to RelNodes > * Have an optimizer rule to optimize Pig group/cogroup into Aggregate > operators > 2. core: > * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL > * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (CALCITE-2921) Allow NULL as argument to function calls
[
https://issues.apache.org/jira/browse/CALCITE-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-2921.
-
Resolution: Fixed
Fixed in
[01477c4|https://github.com/apache/calcite/commit/01477c42ded991aaa7b14cf72a6e4ea1a20edf5e],
thanks for the PR [~pzw2018] and the kindly reminder [~zabetak] !
> Allow NULL as argument to function calls
>
>
> Key: CALCITE-2921
> URL: https://issues.apache.org/jira/browse/CALCITE-2921
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: pengzhiwei
>Assignee: Julian Hyde
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently {{nullif(null,y)}} throws exception in verification. This is
> because translates {{nullif(x,y)}} to a case-when expression, just like "case
> when x = y then null else x". So when "x" is null literal,a exception throws
> out as follow:
> {code:java}
> ELSE clause or at least one THEN clause must be non-NULL
> {code}
> I have test in mysql,"nullif(null,y) works well.So I think we should allow
> this usage of "nullif".
> There are two ways to fix this issue:
> 1) Skip the check for "foundNotNull" in SqlCaseOperator#checkOperandTypes:
>
> {code:java}
> if (!foundNotNull) {
> // according to the sql standard we can not have all of the THEN
> // statements and the ELSE returning null
> if (throwOnFailure) {
> throw callBinding.newError(RESOURCE.mustNotNullInElse());
> }
> return false;
> }{code}
> However, as the comment says, we cannot have all of the THEN and ELSE
> returning null.
> 2) Disable the translation from nullif to case-when and keep "nullif" as it
> is.
> Any suggestion is welcomed,Thanks!
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-2302.
-
Resolution: Fixed
Fixed in
[01477c4|https://github.com/apache/calcite/commit/01477c42ded991aaa7b14cf72a6e4ea1a20edf5e]
!
Thanks for your review [~hyuan], [~julianhyde], and the kindly reminder
[~zabetak] !
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (CALCITE-2325) Calcite can not verify a explicit cast column in some scenario
[
https://issues.apache.org/jira/browse/CALCITE-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-2325.
-
Fix Version/s: 1.21.0
Assignee: Danny Chan
Resolution: Fixed
Fixed in
[01477c4|https://github.com/apache/calcite/commit/01477c42ded991aaa7b14cf72a6e4ea1a20edf5e]
!
> Calcite can not verify a explicit cast column in some scenario
> --
>
> Key: CALCITE-2325
> URL: https://issues.apache.org/jira/browse/CALCITE-2325
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.21.0
>
>
> In now calcite core, there is a test case in agg.ig in line467:
> {code:java}
> select deptno / 2 + 1 as half1, count(*) as c
> from emp
> group by rollup(deptno / 2, gender), rollup(substring(ename FROM 1 FOR
> 1));{code}
> But if we add in an explicit cast:
> {code:java}
> select cast(deptno as decimal) / 2 + 1 as half1, count(*) as c
> from emp
> group by rollup(deptno / 2, gender), rollup(substring(ename FROM 1 FOR
> 1));{code}
> it will throw an error:
>
> {code:java}
> Caused by: org.apache.calcite.runtime.CalciteContextException: From line 1,
> column 13 to line 1, column 18: Expression 'DEPTNO' is not being grouped
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
> at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:810)
> at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:795)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4749)
> at org.apache.calcite.sql.validate.AggChecker.visit(AggChecker.java:117)
> at org.apache.calcite.sql.validate.AggChecker.visit(AggChecker.java:41)
> at org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:344)
> at
> org.apache.calcite.sql.util.SqlBasicVisitor$ArgHandlerImpl.visitChild(SqlBasicVisitor.java:123)
> at org.apache.calcite.sql.SqlOperator.acceptCall(SqlOperator.java:859)
> at org.apache.calcite.sql.validate.AggChecker.visit(AggChecker.java:212)
> at org.apache.calcite.sql.validate.AggChecker.visit(AggChecker.java:41)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138)
> at
> org.apache.calcite.sql.util.SqlBasicVisitor$ArgHandlerImpl.visitChild(SqlBasicVisitor.java:123)
> at org.apache.calcite.sql.SqlOperator.acceptCall(SqlOperator.java:859)
> at org.apache.calcite.sql.validate.AggChecker.visit(AggChecker.java:212)
> at org.apache.calcite.sql.validate.AggChecker.visit(AggChecker.java:41)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138)
> at
> org.apache.calcite.sql.util.SqlBasicVisitor$ArgHandlerImpl.visitChild(SqlBasicVisitor.java:123)
> at org.apache.calcite.sql.SqlOperator.acceptCall(SqlOperator.java:859)
> at org.apache.calcite.sql.validate.AggChecker.visit(AggChecker.java:212)
> at org.apache.calcite.sql.validate.AggChecker.visit(AggChecker.java:41)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138)
> at
> org.apache.calcite.sql.util.SqlBasicVisitor$ArgHandlerImpl.visitChild(SqlBasicVisitor.java:123)
> at org.apache.calcite.sql.SqlAsOperator.acceptCall(SqlAsOperator.java:121)
> at org.apache.calcite.sql.validate.AggChecker.visit(AggChecker.java:212)
> at org.apache.calcite.sql.validate.AggChecker.visit(AggChecker.java:41)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138)
> at
> org.apache.calcite.sql.validate.AggregatingSelectScope.checkAggregateExpr(AggregatingSelectScope.java:231)
> at
> org.apache.calcite.sql.validate.AggregatingSelectScope.validateExpr(AggregatingSelectScope.java:240)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateExpr(SqlValidatorImpl.java:4071)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4045)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3259)
> at
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
> at
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:972)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:948)
> at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:226)
> at
>
[jira] [Commented] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918405#comment-16918405
]
Danny Chan commented on CALCITE-2302:
-
[~julianhyde] Thanks.
There seems to be 2 style type coercions for binary arithmetic with strings:
* For MySQL and Oracle style: coerce all the STRING operand to DOUBLE type
* PostgreSQL and SQL-SERVER style: coerce the STRING operand to the type of the
other operand(if it is a NUMERIC)
The SQL standard[1] says that,
bq. When the declared type of both operands of the addition, subtraction,
multiplication, or division operator is exact numeric, the declared type of the
result is an implementation-defined exact numeric type
So i agree that the PostgreSQL style is more in line with the SQL standard.
Let's choose the PostgreSQL style coercion, that means '9'/2 return INTEGER and
'9'/cast(2.1 as float) returns FLOAT.
For two integers divide, let's keep the return type same as before(9/2 returns
4).
[1] SQL-2011 Subclause 6.27, ""
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3304) SqlValidatorException when use calcite view
[
https://issues.apache.org/jira/browse/CALCITE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917694#comment-16917694
]
Danny Chan commented on CALCITE-3304:
-
This is because you config the FrameworkConfig to match identifier
case-sensitively. To match the id correctly, either to config it be
case-insensitive or add quotes character to the identifier.
> SqlValidatorException when use calcite view
> ---
>
> Key: CALCITE-3304
> URL: https://issues.apache.org/jira/browse/CALCITE-3304
> Project: Calcite
> Issue Type: Bug
>Reporter: Water Cut Off
>Priority: Major
> Attachments: image-2019-08-28-19-15-23-485.png,
> image-2019-08-28-19-16-22-659.png, image-2019-08-28-19-17-40-352.png
>
>
> *{color:#00875a}v_emp:'s sql : select * from emp where empid>10.{color}*
>
> *{color:#de350b}Problem 1：{color}*
> *{color:#de350b}When I use calcite view (as in the blow, v_emp is a view ),
> it occurs such problem, why ?（The test SQL is{color}*
> ”select empid from v_emp where v_emp.empid in(select empid from emp where
> empid=1)”
> *{color:#de350b}）{color}*
> !image-2019-08-28-19-17-40-352.png!
> ---
> My Config is blow, since I have set *CaseSensitive is false.*
> final FrameworkConfig fromworkConfig =
> Frameworks.newConfigBuilder().parserConfig(SqlParser.configBuilder().{color:#de350b}*setCaseSensitive(false)*.{color}build()).defaultSchema(rootSchema).traitDefs(ConventionTraitDef.INSTANCE,
> RelDistributionTraitDef.INSTANCE).build();
>
>
> *{color:#de350b}Problem 2：{color}*
> *{color:#de350b}when I use view, how I can get RelDataType? Thanks a
> lot.{color}*
>
> public class MyViewTable extends AbstractTable implements TranslatableTable {
> private final String vName;
> private final String viewSql;
> public MyViewTable (String vName, String viewSql)
> { super(); this.vName = vName; this.viewSql = viewSql; }
> public String getVName()
> { return vName; }
> @Override
> public RelNode toRel(RelOptTable.ToRelContext context, RelOptTable
> relOptTable)
> { return expandView(context, relOptTable.getRowType(), viewSql).rel; }
> @Override
> {color:#de350b}public RelDataType getRowType(final RelDataTypeFactory
> typeFactory) {{color}
> {color:#de350b} return *???*{color}
> {color:#de350b} }{color}
> @Override
> public Schema.TableType getJdbcTableType()
> { return Schema.TableType.VIEW; }
> private RelRoot expandView(RelOptTable.ToRelContext context, RelDataType
> rowType, String queryString) {
> try {
> final RelRoot root = context.expandView(rowType, queryString, null, null);
> final RelNode rel = RelOptUtil.createCastRel(root.rel, rowType, true);
> // Expand any views
> final RelNode rel2 = rel.accept(new RelShuttleImpl() {
> @Override
> public RelNode visit(TableScan scan) {
> final RelOptTable table = scan.getTable();
> final TranslatableTable translatableTable =
> table.unwrap(TranslatableTable.class);
> if (translatableTable != null)
> { return translatableTable.toRel(context, table); }
> return super.visit(scan);
> }
> });
> return root.withRel(rel2);
> } catch (Exception e)
> { throw new RuntimeException("Error while parsing view definition: " +
> queryString, e); }
> }
> }
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (CALCITE-3304) SqlValidatorException when use calcite view
[
https://issues.apache.org/jira/browse/CALCITE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3304.
-
Resolution: Not A Bug
> SqlValidatorException when use calcite view
> ---
>
> Key: CALCITE-3304
> URL: https://issues.apache.org/jira/browse/CALCITE-3304
> Project: Calcite
> Issue Type: Bug
>Reporter: Water Cut Off
>Priority: Major
> Attachments: image-2019-08-28-19-15-23-485.png,
> image-2019-08-28-19-16-22-659.png, image-2019-08-28-19-17-40-352.png
>
>
> *{color:#00875a}v_emp:'s sql : select * from emp where empid>10.{color}*
>
> *{color:#de350b}Problem 1：{color}*
> *{color:#de350b}When I use calcite view (as in the blow, v_emp is a view ),
> it occurs such problem, why ?（The test SQL is{color}*
> ”select empid from v_emp where v_emp.empid in(select empid from emp where
> empid=1)”
> *{color:#de350b}）{color}*
> !image-2019-08-28-19-17-40-352.png!
> ---
> My Config is blow, since I have set *CaseSensitive is false.*
> final FrameworkConfig fromworkConfig =
> Frameworks.newConfigBuilder().parserConfig(SqlParser.configBuilder().{color:#de350b}*setCaseSensitive(false)*.{color}build()).defaultSchema(rootSchema).traitDefs(ConventionTraitDef.INSTANCE,
> RelDistributionTraitDef.INSTANCE).build();
>
>
> *{color:#de350b}Problem 2：{color}*
> *{color:#de350b}when I use view, how I can get RelDataType? Thanks a
> lot.{color}*
>
> public class MyViewTable extends AbstractTable implements TranslatableTable {
> private final String vName;
> private final String viewSql;
> public MyViewTable (String vName, String viewSql)
> { super(); this.vName = vName; this.viewSql = viewSql; }
> public String getVName()
> { return vName; }
> @Override
> public RelNode toRel(RelOptTable.ToRelContext context, RelOptTable
> relOptTable)
> { return expandView(context, relOptTable.getRowType(), viewSql).rel; }
> @Override
> {color:#de350b}public RelDataType getRowType(final RelDataTypeFactory
> typeFactory) {{color}
> {color:#de350b} return *???*{color}
> {color:#de350b} }{color}
> @Override
> public Schema.TableType getJdbcTableType()
> { return Schema.TableType.VIEW; }
> private RelRoot expandView(RelOptTable.ToRelContext context, RelDataType
> rowType, String queryString) {
> try {
> final RelRoot root = context.expandView(rowType, queryString, null, null);
> final RelNode rel = RelOptUtil.createCastRel(root.rel, rowType, true);
> // Expand any views
> final RelNode rel2 = rel.accept(new RelShuttleImpl() {
> @Override
> public RelNode visit(TableScan scan) {
> final RelOptTable table = scan.getTable();
> final TranslatableTable translatableTable =
> table.unwrap(TranslatableTable.class);
> if (translatableTable != null)
> { return translatableTable.toRel(context, table); }
> return super.visit(scan);
> }
> });
> return root.withRel(rel2);
> } catch (Exception e)
> { throw new RuntimeException("Error while parsing view definition: " +
> queryString, e); }
> }
> }
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3244) RelDecorrelator unable to decorrelate expression with filter and aggregate on top
[
https://issues.apache.org/jira/browse/CALCITE-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917560#comment-16917560
]
Danny Chan commented on CALCITE-3244:
-
Yes, it is the RelDecorrelator that cause the error, not the sub-query rewrite,
so it is not surprise that the three-value-logic enforcing doesn't work, i
would spend some time for the error.
> RelDecorrelator unable to decorrelate expression with filter and aggregate on
> top
> -
>
> Key: CALCITE-3244
> URL: https://issues.apache.org/jira/browse/CALCITE-3244
> Project: Calcite
> Issue Type: Improvement
>Affects Versions: 1.20.0
>Reporter: benj
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Some very useful type of requests currently failed with:
> {code:java}
> SYSTEM ERROR: UnsupportedOperationException: Adding Implicit RowID column is
> not supported for ValuesPrel operator
> {code}
> Examples from DRILL-7050:
> {code:sql}
> select t1.id,
> (select count(t2.id)
> from (
> select 1 as id
> union all
> select 2 as id
> ) t2 where t2.id = t1.id)
> from (
> select 1 as id
> union all
> select 2 as id
> ) t1
> {code}
> {code:sql}
> SELECT t,
> (SELECT count(*) FROM
> (SELECT split(r,' ') AS r FROM
> (SELECT sub.t AS r)) AS x
> ,LATERAL(SELECT $unnest AS u FROM unnest(x.r))
> /* WHERE ... */) t2
> FROM
> (SELECT 'unnest is useful' AS t) sub
> {code}
>
> _Please note that in 1.18 the error for these requests was:_
> {code:java}
> Error: PLAN ERROR: Cannot convert RexNode to equivalent Drill expression.
> RexNode Class: org.apache.calcite.rex.RexCorrelVariable, RexNode Digest: $cor0
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3244) RelDecorrelator unable to decorrelate expression with filter and aggregate on top
[
https://issues.apache.org/jira/browse/CALCITE-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917367#comment-16917367
]
Danny Chan commented on CALCITE-3244:
-
I haven't digged into the failing test case in misc.iq#L709, [~vvysotskyi] can
you help to take a look ?
> RelDecorrelator unable to decorrelate expression with filter and aggregate on
> top
> -
>
> Key: CALCITE-3244
> URL: https://issues.apache.org/jira/browse/CALCITE-3244
> Project: Calcite
> Issue Type: Improvement
>Affects Versions: 1.20.0
>Reporter: benj
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Some very useful type of requests currently failed with:
> {code:java}
> SYSTEM ERROR: UnsupportedOperationException: Adding Implicit RowID column is
> not supported for ValuesPrel operator
> {code}
> Examples from DRILL-7050:
> {code:sql}
> select t1.id,
> (select count(t2.id)
> from (
> select 1 as id
> union all
> select 2 as id
> ) t2 where t2.id = t1.id)
> from (
> select 1 as id
> union all
> select 2 as id
> ) t1
> {code}
> {code:sql}
> SELECT t,
> (SELECT count(*) FROM
> (SELECT split(r,' ') AS r FROM
> (SELECT sub.t AS r)) AS x
> ,LATERAL(SELECT $unnest AS u FROM unnest(x.r))
> /* WHERE ... */) t2
> FROM
> (SELECT 'unnest is useful' AS t) sub
> {code}
>
> _Please note that in 1.18 the error for these requests was:_
> {code:java}
> Error: PLAN ERROR: Cannot convert RexNode to equivalent Drill expression.
> RexNode Class: org.apache.calcite.rex.RexCorrelVariable, RexNode Digest: $cor0
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908903#comment-16908903
]
Danny Chan edited comment on CALCITE-2302 at 8/28/19 2:58 AM:
--
Thanks so much for your review [~hyuan] !
I have add a new SqlConformance for PostgreSql and SqlServer which seem the
only engine that returns integer for 2 integers division. I checked again for
the behavior:
* Hive and Spark all return double, Hive also have a "A DIV B" function to
return integer
* PostgreSql and SQL_SERVER returns integer.
* Mysql returns double while it has a DIV() function to return integer
* Oracle returns double, but have no function to return integer
was (Author: danny0405):
Thanks so much for your review [~hyuan] !
I have add a new SqlConformance for PostgreSql and SqlServer which seem the
only engine that returns integer for 2 integers division. I checked again for
the behavior:
* Hive and Spark all return double, Hive also have a "A DIV B" function to
return integer
* PostgreSql and SQL_SERVER returns integer.
* Mysql returns double while it has a DIV() function to return integer
* Oracle returns double, it also has a DIV() function to return integer
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3244) RelDecorrelator unable to decorrelate expression with filter and aggregate on top
[
https://issues.apache.org/jira/browse/CALCITE-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917355#comment-16917355
]
Danny Chan commented on CALCITE-3244:
-
Thanks [~vvysotskyi], i have fixed the test case you pasted.
> RelDecorrelator unable to decorrelate expression with filter and aggregate on
> top
> -
>
> Key: CALCITE-3244
> URL: https://issues.apache.org/jira/browse/CALCITE-3244
> Project: Calcite
> Issue Type: Improvement
>Affects Versions: 1.20.0
>Reporter: benj
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Some very useful type of requests currently failed with:
> {code:java}
> SYSTEM ERROR: UnsupportedOperationException: Adding Implicit RowID column is
> not supported for ValuesPrel operator
> {code}
> Examples from DRILL-7050:
> {code:sql}
> select t1.id,
> (select count(t2.id)
> from (
> select 1 as id
> union all
> select 2 as id
> ) t2 where t2.id = t1.id)
> from (
> select 1 as id
> union all
> select 2 as id
> ) t1
> {code}
> {code:sql}
> SELECT t,
> (SELECT count(*) FROM
> (SELECT split(r,' ') AS r FROM
> (SELECT sub.t AS r)) AS x
> ,LATERAL(SELECT $unnest AS u FROM unnest(x.r))
> /* WHERE ... */) t2
> FROM
> (SELECT 'unnest is useful' AS t) sub
> {code}
>
> _Please note that in 1.18 the error for these requests was:_
> {code:java}
> Error: PLAN ERROR: Cannot convert RexNode to equivalent Drill expression.
> RexNode Class: org.apache.calcite.rex.RexCorrelVariable, RexNode Digest: $cor0
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Issue Comment Deleted] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-2302:
Comment: was deleted
(was: [~julianhyde] Does the latest commit solve your review objections ? I'm
planning to merge this PR if there are no more comments in 24 hours.)
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914070#comment-16914070
]
Danny Chan edited comment on CALCITE-2302 at 8/28/19 1:32 AM:
--
I have made the latest change in commit
[94134d5|https://github.com/apache/calcite/pull/706/commits/94134d5dd6a9c38cc22ff74bdd05a31d76e41b29]
!
Instead of leveraging the existing RelDataTypeSystem.deriveDecimalDivideType
method, i add a new SqlReturnTypeInference named NUMERIC_QUOTIENT chained into
the exists return type inference of "DIVIDE" operator.
Also i modified the expression to make them returns double(2 QuidemTest changes
because of this change, their results become more accurate).
was (Author: danny0405):
I have made the latest change in commit
[7b03ac7|https://github.com/apache/calcite/pull/706/commits/7b03ac7f466c248a0765447ce4a18c9d9393aa95]
!
Instead of leveraging the existing RelDataTypeSystem.deriveDecimalDivideType
method, i add a new SqlReturnTypeInference named NUMERIC_QUOTIENT chained into
the exists return type inference of "DIVIDE" operator.
Also i modified the expression to make them returns double(2 QuidemTest changes
because of this change, their results become more accurate).
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3244) RelDecorrelator unable to decorrelate expression with filter and aggregate on top
[
https://issues.apache.org/jira/browse/CALCITE-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916709#comment-16916709
]
Danny Chan commented on CALCITE-3244:
-
[~vvysotskyi] I have applied a patch to fix the decorrelation.
But with the fix, there is a breaking test in misc.iq#L709, the original test
passes because just because it can not decorrelate values expression, if the
change the values to project, the test still fails, so i think this is another
fix and should fire the fix in another issue.
> RelDecorrelator unable to decorrelate expression with filter and aggregate on
> top
> -
>
> Key: CALCITE-3244
> URL: https://issues.apache.org/jira/browse/CALCITE-3244
> Project: Calcite
> Issue Type: Improvement
>Affects Versions: 1.20.0
>Reporter: benj
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Some very useful type of requests currently failed with:
> {code:java}
> SYSTEM ERROR: UnsupportedOperationException: Adding Implicit RowID column is
> not supported for ValuesPrel operator
> {code}
> Examples from DRILL-7050:
> {code:sql}
> select t1.id,
> (select count(t2.id)
> from (
> select 1 as id
> union all
> select 2 as id
> ) t2 where t2.id = t1.id)
> from (
> select 1 as id
> union all
> select 2 as id
> ) t1
> {code}
> {code:sql}
> SELECT t,
> (SELECT count(*) FROM
> (SELECT split(r,' ') AS r FROM
> (SELECT sub.t AS r)) AS x
> ,LATERAL(SELECT $unnest AS u FROM unnest(x.r))
> /* WHERE ... */) t2
> FROM
> (SELECT 'unnest is useful' AS t) sub
> {code}
>
> _Please note that in 1.18 the error for these requests was:_
> {code:java}
> Error: PLAN ERROR: Cannot convert RexNode to equivalent Drill expression.
> RexNode Class: org.apache.calcite.rex.RexCorrelVariable, RexNode Digest: $cor0
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-2973) Allow theta joins that have equi conditions to be executed using a hash join algorithm
[ https://issues.apache.org/jira/browse/CALCITE-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916465#comment-16916465 ] Danny Chan commented on CALCITE-2973: - CALCITE-2302 and CALCITE-1581 are almost ready to be merged, i think it's okey to push this patch into release-1.21 if you think it is in enough good shape. > Allow theta joins that have equi conditions to be executed using a hash join > algorithm > -- > > Key: CALCITE-2973 > URL: https://issues.apache.org/jira/browse/CALCITE-2973 > Project: Calcite > Issue Type: New Feature > Components: core >Affects Versions: 1.19.0 >Reporter: Lai Zhou >Priority: Minor > Labels: pull-request-available > Time Spent: 8h 20m > Remaining Estimate: 0h > > Now the EnumerableMergeJoinRule only supports an inner and equi join. > If users make a theta-join query for a large dataset (such as 1*1), > the nested-loop join process will take dozens of time than the sort-merge > join process . > So if we can apply merge-join or hash-join rule for a theta join, it will > improve the performance greatly. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (CALCITE-3210) JDBC adapter should generate "CAST(NULL AS type)" rather than "NULL"
[
https://issues.apache.org/jira/browse/CALCITE-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3210.
-
Fix Version/s: (was: next)
1.21.0
Resolution: Fixed
Fixed in
[e863294|https://github.com/apache/calcite/commit/e863294ccfbed9dd520c999f75ed0bbe03f9fb1d],
thanks for your PR, [~wangweidong] !
> JDBC adapter should generate "CAST(NULL AS type)" rather than "NULL"
>
>
> Key: CALCITE-3210
> URL: https://issues.apache.org/jira/browse/CALCITE-3210
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
> Environment: mac os
>Reporter: Wang Weidong
>Assignee: Wang Weidong
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 9h 40m
> Remaining Estimate: 0h
>
> I have as sql as follows:
> {code:java}
> select cast(null as varchar) as a{code}
> The process step is :
> # parse sql to SqlNode;
> # convert SqlNode to RelNode;
> # convert RelNode to SqlNode;
> # convert SqlNode to string.
> The result string is:
> {code:java}
> "SELECT NULL AS `A`\n"
> + "FROM (VALUES (0)) AS `t` (`ZERO`)"{code}
> Finally, I found that the result string is absent of cast statement for
> "null" which causes the result sql to be invalid.
> Actually, the result I expect is like:
> {code:java}
> "SELECT CAST(NULL AS VARCHAR) AS `A`\n"
> + "FROM (VALUES (0)) AS `t` (`ZERO`)"{code}
>
> Testing code is as follows
> {code:java}
> // code placeholder
> @Test
> public void testSeletNull() throws SqlParseException {
> String sql = "select cast(null as varchar) as a";
> SqlNode sqlNode = parserContext.parseStmt(sql);
> RelNode relNode =
> parserContext.getSqlToRelConverter().convertQuery(sqlNode, true, true).rel;
> SqlNode sqlNodeNew = toSqlNode(relNode);
> Assert.assertEquals(sqlNodeNew.toString(), "SELECT NULL AS `A`\n"
> + "FROM (VALUES (0)) AS `t` (`ZERO`)");
> }
> private static SqlNode toSqlNode(RelNode root) {
> SqlDialect dialect = MysqlSqlDialect.DEFAULT;
> RelToSqlConverter converter = new RelToSqlConverter(dialect == null ?
> dialect : dialect);
> return converter.visitChild(0, root).asStatement();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-482) Implement sql and planner hints
[ https://issues.apache.org/jira/browse/CALCITE-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915532#comment-16915532 ] Danny Chan commented on CALCITE-482: Thanks, [~hyuan] and [~Chunwei Lei] i have attached the design doc, can you help to review the design ? Maybe we should have a discussion on the doc, or under this issue, either is ok. > Implement sql and planner hints > --- > > Key: CALCITE-482 > URL: https://issues.apache.org/jira/browse/CALCITE-482 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.0.0-incubating >Reporter: Vladimir Sitnikov >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > Query optimizer can never be perfect, so it makes sense to implement hints to > allow user better control the execution. > For instance: "never merge this subquery with others" (`/+ no_merge/`), > "treat those tables as leading ones" (`/*+ leading */`), etc. > Hints would enable predictable performance and the planning time would be > improved as well. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915414#comment-16915414
]
Danny Chan commented on CALCITE-2302:
-
[~julianhyde] Does the latest commit solve your review objections ? I'm
planning to merge this PR if there are no more comments in 24 hours.
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3282) Make every SqlDialect unparse their own data type
[ https://issues.apache.org/jira/browse/CALCITE-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915407#comment-16915407 ] Danny Chan commented on CALCITE-3282: - The key point is that we don't want the SqlTypeNameSpec/SqlDataTypeSpec to have a mess of branches to handle kinds of sql dialects unparse logic. Let's keep them clean from sql dialects and do these "dialect" thing in SqlDialect itself. > Make every SqlDialect unparse their own data type > - > > Key: CALCITE-3282 > URL: https://issues.apache.org/jira/browse/CALCITE-3282 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: feng huang >Priority: Minor > > Every database might have different type or same type but different type > name, therefore making every SqlDialect unparse their own data type is a > suitable way. > For example, there is a sql “select cast(col as int) from table” change to > hive sql "select cast(col as integer) from table", but "integer" is not > allowed in hive. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914070#comment-16914070
]
Danny Chan edited comment on CALCITE-2302 at 8/24/19 2:31 AM:
--
I have made the latest change in commit
[7b03ac7|https://github.com/apache/calcite/pull/706/commits/7b03ac7f466c248a0765447ce4a18c9d9393aa95]
!
Instead of leveraging the existing RelDataTypeSystem.deriveDecimalDivideType
method, i add a new SqlReturnTypeInference named NUMERIC_QUOTIENT chained into
the exists return type inference of "DIVIDE" operator.
Also i modified the expression to make them returns double(2 QuidemTest changes
because of this change, their results become more accurate).
was (Author: danny0405):
I have made the latest change in commit
[7b03ac7|https://github.com/apache/calcite/pull/706/commits/7b03ac7f466c248a0765447ce4a18c9d9393aa95]
!
Instead of leveraging the existing RelDataTypeSystem.deriveDecimalDivideType
method, i add a new SqlReturnTypeInference named EXACT_INTEGER_QUOTIENT chained
into the exists return type inference of "DIVIDE" operator.
Also i modified the expression to make them returns double(2 QuidemTest changes
because of this change, their results become more accurate).
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3282) Make every SqlDialect unparse their own data type
[ https://issues.apache.org/jira/browse/CALCITE-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914095#comment-16914095 ] Danny Chan commented on CALCITE-3282: - Add a new method SqlDialect#unparseDataType seems a right way to go, then in the SqlDialect#unparseCall, you should have a default implementation to unparse the Cast operator, make the type unparse to invoke the new method #unparseDataType. > Make every SqlDialect unparse their own data type > - > > Key: CALCITE-3282 > URL: https://issues.apache.org/jira/browse/CALCITE-3282 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: feng huang >Priority: Minor > > Every database might have different type or same type but different type > name, therefore making every SqlDialect unparse their own data type is a > suitable way. > For example, there is a sql “select cast(col as int) from table” change to > hive sql "select cast(col as integer) from table", but "integer" is not > allowed in hive. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914070#comment-16914070
]
Danny Chan commented on CALCITE-2302:
-
I have made the latest change in commit
[7b03ac7|https://github.com/apache/calcite/pull/706/commits/7b03ac7f466c248a0765447ce4a18c9d9393aa95]
!
Instead of leveraging the existing RelDataTypeSystem.deriveDecimalDivideType
method, i add a new SqlReturnTypeInference named EXACT_INTEGER_QUOTIENT chained
into the exists return type inference of "DIVIDE" operator.
Also i modified the expression to make them returns double(2 QuidemTest changes
because of this change, their results become more accurate).
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913843#comment-16913843
]
Danny Chan edited comment on CALCITE-2302 at 8/23/19 3:46 AM:
--
Thanks, [~julianhyde] I kind of agree with your point that the "DIVIDE"
operator is the role to decide what kind of data it should return, not the
implicit type casts. Even though the implicit casts can implement as the same
behavior with configurable sql dialects.
Personally I have no strong objections on leverage the return type inferring of
RelDataTypeSystem.deriveDecimalDivideType method.
Let's give up the thoughts that we should keep the "DIVIDE" synced and
configurable with different sql dialects.
was (Author: danny0405):
Thanks, [~julianhyde] I kind of agree with your point that the "DIVIDE"
operator is the role to decide what kind of data it should return, not the
implicit type casts. Even though the implicit casts can implement as the same
behavior with configurable sql dialects.
Personally I have no strong objections on leverage the return type inferring of
RelDataTypeSystem.deriveDecimalDivideType method, just one question:
# How to support different sql dialects ? Because PostgreSQL and SQL-SERVER
return integer and the others return double ? Especially for our JDBC
connectors, do we need add a control flag or something ?
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3282) Make every SqlDialect unparse their own data type
[ https://issues.apache.org/jira/browse/CALCITE-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913878#comment-16913878 ] Danny Chan commented on CALCITE-3282: - The SqlTypeNameSpec is a component of SqlDataTypeSpec to describe the sql type name (with attributes like type name, precision/scale, charSet), the SqlTypeNameSpec.unparse is invoked by SqlDataTypeSpec.unparse. The SqlDataTypeSpec.unparse method is inherit from SqlNode.unparse, i think of 2 way to support your idea: * extend SqlNode.unparse to support sql dialect which is a huge change i don't suggest to do. * In the different, SqlDialect, match the SqlDataTypeSpec and unparse it specifically, we better do not modify the SqlDataTypeSpec and SqlTypeNameSpec, just put the unparse logic in SqlDialect. (We already did so for some rex call, i.e. the cast operation). > Make every SqlDialect unparse their own data type > - > > Key: CALCITE-3282 > URL: https://issues.apache.org/jira/browse/CALCITE-3282 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: feng huang >Priority: Minor > > Every database might have different type or same type but different type > name, therefore making every SqlDialect parse their own data type is a > suitable way. > for example, “select cast(col as int) from table” change to hive sql "select > cast(col as integer) from table", but "integer" is not allowed in hive. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913843#comment-16913843
]
Danny Chan commented on CALCITE-2302:
-
Thanks, [~julianhyde] I kind of agree with your point that the "DIVIDE"
operator is the role to decide what kind of data it should return, not the
implicit type casts. Even though the implicit casts can implement as the same
behavior with configurable sql dialects.
Personally I have no strong objections on leverage the return type inferring of
RelDataTypeSystem.deriveDecimalDivideType method, just one question:
# How to support different sql dialects ? Because PostgreSQL and SQL-SERVER
return integer and the others return double ? Especially for our JDBC
connectors, do we need add a control flag or something ?
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3282) make every SqlDialect parse their own data type
[ https://issues.apache.org/jira/browse/CALCITE-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913252#comment-16913252 ] Danny Chan commented on CALCITE-3282: - Sounds reasonable, but i can not figure out a proper way to fix "every" sql dialect, there are kinds of sql dialects we supported, every sql dialect have they own unparse logic, so, to keep all these type sync with every dialect seems not an easy work. We may need to make sure what sql dialect we want to fix first then make the type list that need fix. > make every SqlDialect parse their own data type > --- > > Key: CALCITE-3282 > URL: https://issues.apache.org/jira/browse/CALCITE-3282 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: feng huang >Priority: Minor > > Every database might have different type or same type but different type > name, therefore making every SqlDialect parse their own data type is a > suitable way. > for example, “select cast(col as int) from table” change to hive sql "select > cast(col as integer) from table", but "integer" is not allowed in hive. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.
[
https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16912928#comment-16912928
]
Danny Chan commented on CALCITE-3115:
-
Reviewing now ..
> Cannot add JdbcRules which have different JdbcConvention to same
> VolcanoPlanner's RuleSet.
> --
>
> Key: CALCITE-3115
> URL: https://issues.apache.org/jira/browse/CALCITE-3115
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.19.0
>Reporter: TANG Wen-hui
>Assignee: Igor Guzenko
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> When we use Calcite via JDBC to run a sql which involves two difference jdbc
> schema:
> {code:java}
> select * from (select "a",max("b") as max_b, sum("c") as sum_c from
> "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having
> "a" > 100 and max("b") < 20 limit 10) t union select "a", "b","c" from
> "test2"."temp2" group by "a","b","c"
> {code}
> the sql get a plan like that:
> {code:java}
> EnumerableUnion(all=[false])
> JdbcToEnumerableConverter
> JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4])
> JdbcSort(fetch=[10])
> JdbcFilter(condition=[<(CAST($3):BIGINT, 20)])
> JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)])
> JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0,
> 100))])
> JdbcTableScan(table=[[test, temp]])
> EnumerableAggregate(group=[{0, 1, 2}])
> JdbcToEnumerableConverter
> JdbcTableScan(table=[[test2, temp2]])
> {code}
> And the EnumerableAggregate for table test2.temp2 cannot be converted to
> JdbcAggregate.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16912840#comment-16912840
]
Danny Chan commented on CALCITE-2302:
-
[~julianhyde] Does my reply answer your questions ? I'm planning to merge this
PR if there are no more comments in 24 hours.
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16912839#comment-16912839
]
Danny Chan commented on CALCITE-2302:
-
For "9/2 returns 4.5", we indeed do the type coercion because we need to coerce
all the operands of "/" to double. So this change definitely belongs to the
implicit type coercion scope.
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911987#comment-16911987
]
Danny Chan commented on CALCITE-2302:
-
[~julianhyde] Personally i propose to add this policy to SqlConformance,
because it is very about the execution behavior of a SqlDialect, the
RelDataTypeSystem defines some type deriving behaviors of the entire type
system in methods like "deriveXXX", but strictly to say, "two integers division
returns what kind of data" not only is relative with "what sql type we should
return" but "what data we should return".
We can say that "9/2 should return double", then 4.5 and 4.0 can both be seen
as double type. For an implicit type coercion, what we really changed is the
runtime/execution behavior, not only the type inferring.
Another reason is that RelDataTypeSystem is about Calcite type system behaviors
which are not designed pluggable for all kinds of SQL dialects.
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (CALCITE-3213) Support complex type expressions for SqlDataTypeSpec
[
https://issues.apache.org/jira/browse/CALCITE-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3213.
-
Resolution: Fixed
> Support complex type expressions for SqlDataTypeSpec
> -
>
> Key: CALCITE-3213
> URL: https://issues.apache.org/jira/browse/CALCITE-3213
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.21.0
>
>
> We should support nested struct type like:
> {code:sql}
> ROW(
> foo NUMBER(5, 2) NOT NULL,
> rec ROW(b BOOLEAN, i MyUDT NOT NULL))
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3271) TVF windowing and EMIT syntax support in Calcite
[ https://issues.apache.org/jira/browse/CALCITE-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan updated CALCITE-3271: Issue Type: Improvement (was: Bug) > TVF windowing and EMIT syntax support in Calcite > > > Key: CALCITE-3271 > URL: https://issues.apache.org/jira/browse/CALCITE-3271 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > > Copied from the mailing list: > Calcite has not implemented the syntax in that paper. I would support an > effort to add it (unsurprising, since I am a co-author of that paper). > EMIT STREAM is equivalent to the current SELECT STREAM syntax. > There is no equivalent in current Calcite of the EMIT AFTER WATERMARK, or > EMIT STREAM AFTER DELAY. > HOP, TUMBLE and SESSION are supported in Calcite’s SQL parser, but following > the paper would be replaced with a table function call. We could need to add > HOP, TUMBLE and SESSION table functions. We would also need to make the > system aware of how watermarks flow through these table functions (an area > that the paper does not go into). > Julian -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3271) TVF windowing and EMIT syntax support in Calcite
Danny Chan created CALCITE-3271: --- Summary: TVF windowing and EMIT syntax support in Calcite Key: CALCITE-3271 URL: https://issues.apache.org/jira/browse/CALCITE-3271 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.20.0 Reporter: Danny Chan Assignee: Danny Chan Copied from the mailing list: Calcite has not implemented the syntax in that paper. I would support an effort to add it (unsurprising, since I am a co-author of that paper). EMIT STREAM is equivalent to the current SELECT STREAM syntax. There is no equivalent in current Calcite of the EMIT AFTER WATERMARK, or EMIT STREAM AFTER DELAY. HOP, TUMBLE and SESSION are supported in Calcite’s SQL parser, but following the paper would be replaced with a table function call. We could need to add HOP, TUMBLE and SESSION table functions. We would also need to make the system aware of how watermarks flow through these table functions (an area that the paper does not go into). Julian -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (CALCITE-3251) BinaryExpression evaluate method support Long type.
[ https://issues.apache.org/jira/browse/CALCITE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan updated CALCITE-3251: Component/s: core Affects Version/s: 1.19.0 1.20.0 > BinaryExpression evaluate method support Long type. > --- > > Key: CALCITE-3251 > URL: https://issues.apache.org/jira/browse/CALCITE-3251 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.19.0, 1.20.0 >Reporter: xzh_dz >Assignee: Danny Chan >Priority: Minor > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 4h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (CALCITE-3265) Remove useless code in ExpressionTest
[ https://issues.apache.org/jira/browse/CALCITE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan updated CALCITE-3265: Fix Version/s: 1.21.0 Affects Version/s: 1.20.0 > Remove useless code in ExpressionTest > - > > Key: CALCITE-3265 > URL: https://issues.apache.org/jira/browse/CALCITE-3265 > Project: Calcite > Issue Type: Wish >Affects Versions: 1.20.0 >Reporter: xzh_dz >Assignee: Danny Chan >Priority: Minor > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (CALCITE-3265) Remove useless code in ExpressionTest
[ https://issues.apache.org/jira/browse/CALCITE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan updated CALCITE-3265: Summary: Remove useless code in ExpressionTest (was: remove useless code) > Remove useless code in ExpressionTest > - > > Key: CALCITE-3265 > URL: https://issues.apache.org/jira/browse/CALCITE-3265 > Project: Calcite > Issue Type: Wish >Reporter: xzh_dz >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (CALCITE-3265) Remove useless code in ExpressionTest
[ https://issues.apache.org/jira/browse/CALCITE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3265. - Assignee: Danny Chan Resolution: Fixed Fixed in [5a42812|https://github.com/apache/calcite/commit/5a42812832e51a7551a793ac75160188e06668e6], thanks for your PR, [~xzh_dz] ! > Remove useless code in ExpressionTest > - > > Key: CALCITE-3265 > URL: https://issues.apache.org/jira/browse/CALCITE-3265 > Project: Calcite > Issue Type: Wish >Reporter: xzh_dz >Assignee: Danny Chan >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (CALCITE-3251) BinaryExpression evaluate method support Long type.
[ https://issues.apache.org/jira/browse/CALCITE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3251. - Fix Version/s: 1.21.0 Assignee: Danny Chan Resolution: Fixed Fixed in [5a42812|https://github.com/apache/calcite/commit/5a42812832e51a7551a793ac75160188e06668e6], thanks for your PR, [~xzh_dz] ! > BinaryExpression evaluate method support Long type. > --- > > Key: CALCITE-3251 > URL: https://issues.apache.org/jira/browse/CALCITE-3251 > Project: Calcite > Issue Type: Bug >Reporter: xzh_dz >Assignee: Danny Chan >Priority: Minor > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 4h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (CALCITE-3267) Remove method SqlDataTypeSpec#deriveType(RelDataTypefactory)
[ https://issues.apache.org/jira/browse/CALCITE-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3267. - Resolution: Fixed Fixed in [6b600e4|https://github.com/apache/calcite/commit/6b600e4bd13d130a4ea48ab9b8451d81ed095da0] ! > Remove method SqlDataTypeSpec#deriveType(RelDataTypefactory) > > > Key: CALCITE-3267 > URL: https://issues.apache.org/jira/browse/CALCITE-3267 > Project: Calcite > Issue Type: Sub-task > Components: core >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > As discussed in CALCITE-3250, we should always derive data type from > SqlValidator instead of RelDataTypeFactory. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-3213) Support complex type expressions for SqlDataTypeSpec
[
https://issues.apache.org/jira/browse/CALCITE-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16910954#comment-16910954
]
Danny Chan commented on CALCITE-3213:
-
Thanks Julian, the change list you proposed is much better and accurate, a big
+1 !
> Support complex type expressions for SqlDataTypeSpec
> -
>
> Key: CALCITE-3213
> URL: https://issues.apache.org/jira/browse/CALCITE-3213
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.21.0
>
>
> We should support nested struct type like:
> {code:sql}
> ROW(
> foo NUMBER(5, 2) NOT NULL,
> rec ROW(b BOOLEAN, i MyUDT NOT NULL))
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (CALCITE-3269) Returns integer for VARCHAR and INT division of PostgreSql dialect
Danny Chan created CALCITE-3269: --- Summary: Returns integer for VARCHAR and INT division of PostgreSql dialect Key: CALCITE-3269 URL: https://issues.apache.org/jira/browse/CALCITE-3269 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.21.0 Reporter: Danny Chan Assignee: Danny Chan We are planning to fix this for PostgreSQL, but maybe not in 1.21.0. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16910144#comment-16910144
]
Danny Chan edited comment on CALCITE-2302 at 8/19/19 2:59 AM:
--
Thanks [~zabetak], most of the sql contexts that need implicit type coercion
are common in the popular DB engines, what we need to be caution is the
different behaviors of some cast cases. We can fix them for specific dialects
if we find any. But AFAIK, this kind of cases are really few.
Because almost all the DB engines that support implicit type coercion default
enable the casts, we should also follow this to default enable type coercion
for Calcite.
was (Author: danny0405):
Thanks [~zabetak], most of the sql contexts that need implicit type coercion
are common in the popular DB engines, what we need to be caution is the
different behaviors of some cast cases. We can fix them for specific dialects
if we find any. But AFAIK, this kind of cases are really few.
But because almost all the DB engines that support implicit type coercion
default enable the casts, we should also follow this to default enable type
coercion for Calcite.
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908903#comment-16908903
]
Danny Chan edited comment on CALCITE-2302 at 8/19/19 2:55 AM:
--
Thanks so much for your review [~hyuan] !
I have add a new SqlConformance for PostgreSql and SqlServer which seem the
only engine that returns integer for 2 integers division. I checked again for
the behavior:
* Hive and Spark all return double, Hive also have a "A DIV B" function to
return integer
* PostgreSql and SQL_SERVER returns integer.
* Mysql returns double while it has a DIV() function to return integer
* Oracle returns double, it also has a DIV() function to return integer
was (Author: danny0405):
Thanks so much for your review [~hyuan] !
I have add a new SqlConformance for PostgreSql and SqlServer which seem the
only engine that returns integer for 2 integers division. I checked again for
the behavior:
* Hive and Spark all return double, Hive also have a "A DIV B" function to
return integer
* PostgreSql and SQL_SERVER returns integer.
* Mysql returns double while it has a DIV() function to return integer
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16910144#comment-16910144
]
Danny Chan edited comment on CALCITE-2302 at 8/19/19 2:52 AM:
--
Thanks [~zabetak], most of the sql contexts that need implicit type coercion
are common in the popular DB engines, what we need to be caution is the
different behaviors of some cast cases. We can fix them for specific dialects
if we find any. But AFAIK, this kind of cases are really few.
But because almost all the DB engines that support implicit type coercion
default enable the casts, we should also follow this to default enable type
coercion for Calcite.
was (Author: danny0405):
Thanks [~zabetak], most of the sql contexts that need implicit type coercion
are common in the popular DB engines, what we need to be caution is the
different behaviors of some cast cases. We can fix them for specific dialects
if we find any. But AFAIK, this kind of cases are really few.
I think we should keep the casts default enabled, which is also adopted by all
the other DB engines.
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16910144#comment-16910144
]
Danny Chan commented on CALCITE-2302:
-
Thanks [~zabetak], most of the sql contexts that need implicit type coercion
are common in the popular DB engines, what we need to be caution is the
different behaviors of some cast cases. We can fix them for specific dialects
if we find any. But AFAIK, this kind of cases are really few.
I think we should keep the casts default enabled, which is also adopted by all
the other DB engines.
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
