[jira] [Created] (CALCITE-3433) EQUALS operator between date/timestamp types returns false if the type is nullable
jiezouSH created CALCITE-3433: - Summary: EQUALS operator between date/timestamp types returns false if the type is nullable Key: CALCITE-3433 URL: https://issues.apache.org/jira/browse/CALCITE-3433 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.21.0, 1.20.0, 1.19.0, 1.18.0, 1.17.0 Reporter: jiezouSH sql select time0=time1 from (select timestamp'2000-12-30 21:07:32'as time0,timestamp'2000-12-30 21:07:32'as time1 union all select cast(null as timestamp) as time0,cast(null as timestamp) as time1) calcs answer is false but sql select time0=time1 from (select timestamp'2000-12-30 21:07:32'as time0,timestamp'2000-12-30 21:07:32'as time1) calcs answer is true -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3427) Some subquery correlated cases are not fully implemented
[
https://issues.apache.org/jira/browse/CALCITE-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955739#comment-16955739
]
Chunwei Lei commented on CALCITE-3427:
--
It would be great if you can provide a test to reproduce it.
> Some subquery correlated cases are not fully implemented
>
>
> Key: CALCITE-3427
> URL: https://issues.apache.org/jira/browse/CALCITE-3427
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
>Reporter: liuzonghao
>Priority: Major
> Fix For: next
>
>
> {code:java}
> //代码占位符
> // for correlated case queries such as
> //
> // select e.deptno, e.deptno < some (
> // select deptno from emp where emp.name = e.name) as v
> // from emp as e
> //
> // becomes
> //
> // select e.deptno,
> // case
> // when indicator is null then false // sub-query is empty for
> corresponding corr value
> // when q.c = 0 then false // sub-query is empty
> // when (e.deptno < q.m) is true then true
> // when q.c > q.d then unknown // sub-query has at least one null
> // else e.deptno < q.m
> // end as v
> // from emp as e
> // left outer join (
> // select name, max(deptno) as m, count(*) as c, count(deptno) as d,
> // "alwaysTrue" as indicator
> // from emp group by name) as q on e.name = q.name
> Set varsUsed = RelOptUtil.getVariablesUsed(e.rel);
> builder.push(e.rel)
> .aggregate(builder.groupKey(),
> builder.aggregateCall(minMax, builder.field(0)).as("m"),
> builder.count(false, "c"),
> builder.count(false, "d", builder.field(0)));
> final List parentQueryFields = new ArrayList<>();
> parentQueryFields.addAll(builder.fields());
> String indicator = "trueLiteral";
> parentQueryFields.add(builder.alias(literalTrue, indicator));
> builder.project(parentQueryFields).as("q");
> builder.join(JoinRelType.LEFT, literalTrue, variablesSet);
> caseRexNode = builder.call(SqlStdOperatorTable.CASE,
> builder.call(SqlStdOperatorTable.IS_NULL,
> builder.field("q", indicator)),
> literalFalse,
> builder.call(SqlStdOperatorTable.EQUALS, builder.field("q", "c"),
> builder.literal(0)),
> literalFalse,
> builder.call(SqlStdOperatorTable.IS_TRUE,
> builder.call(RelOptUtil.op(op.comparisonKind, null),
> e.operands.get(0), builder.field("q", "m"))),
> literalTrue,
> builder.call(SqlStdOperatorTable.GREATER_THAN,
> builder.field("q", "c"), builder.field("q", "d")),
> literalUnknown,
> builder.call(RelOptUtil.op(op.comparisonKind, null),
> e.operands.get(0), builder.field("q", "m")));
> {code}
> implementation code is in SubQueryRemoveRule.rewriteSome method. the groupKey
> and join condition is lacked.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3427) Some subquery correlated cases are not fully implemented
[
https://issues.apache.org/jira/browse/CALCITE-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei updated CALCITE-3427:
-
Summary: Some subquery correlated cases are not fully implemented (was:
some subquery correlated case isn't fully implemented)
> Some subquery correlated cases are not fully implemented
>
>
> Key: CALCITE-3427
> URL: https://issues.apache.org/jira/browse/CALCITE-3427
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
>Reporter: liuzonghao
>Priority: Major
> Fix For: next
>
>
> {code:java}
> //代码占位符
> // for correlated case queries such as
> //
> // select e.deptno, e.deptno < some (
> // select deptno from emp where emp.name = e.name) as v
> // from emp as e
> //
> // becomes
> //
> // select e.deptno,
> // case
> // when indicator is null then false // sub-query is empty for
> corresponding corr value
> // when q.c = 0 then false // sub-query is empty
> // when (e.deptno < q.m) is true then true
> // when q.c > q.d then unknown // sub-query has at least one null
> // else e.deptno < q.m
> // end as v
> // from emp as e
> // left outer join (
> // select name, max(deptno) as m, count(*) as c, count(deptno) as d,
> // "alwaysTrue" as indicator
> // from emp group by name) as q on e.name = q.name
> Set varsUsed = RelOptUtil.getVariablesUsed(e.rel);
> builder.push(e.rel)
> .aggregate(builder.groupKey(),
> builder.aggregateCall(minMax, builder.field(0)).as("m"),
> builder.count(false, "c"),
> builder.count(false, "d", builder.field(0)));
> final List parentQueryFields = new ArrayList<>();
> parentQueryFields.addAll(builder.fields());
> String indicator = "trueLiteral";
> parentQueryFields.add(builder.alias(literalTrue, indicator));
> builder.project(parentQueryFields).as("q");
> builder.join(JoinRelType.LEFT, literalTrue, variablesSet);
> caseRexNode = builder.call(SqlStdOperatorTable.CASE,
> builder.call(SqlStdOperatorTable.IS_NULL,
> builder.field("q", indicator)),
> literalFalse,
> builder.call(SqlStdOperatorTable.EQUALS, builder.field("q", "c"),
> builder.literal(0)),
> literalFalse,
> builder.call(SqlStdOperatorTable.IS_TRUE,
> builder.call(RelOptUtil.op(op.comparisonKind, null),
> e.operands.get(0), builder.field("q", "m"))),
> literalTrue,
> builder.call(SqlStdOperatorTable.GREATER_THAN,
> builder.field("q", "c"), builder.field("q", "d")),
> literalUnknown,
> builder.call(RelOptUtil.op(op.comparisonKind, null),
> e.operands.get(0), builder.field("q", "m")));
> {code}
> implementation code is in SubQueryRemoveRule.rewriteSome method. the groupKey
> and join condition is lacked.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3415) Supports REGEXP_SUBSTR function
[
https://issues.apache.org/jira/browse/CALCITE-3415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955730#comment-16955730
]
Pranay Parmar commented on CALCITE-3415:
[~amaliujia] Sure. That would help a lot. Thank you so much.
> Supports REGEXP_SUBSTR function
> ---
>
> Key: CALCITE-3415
> URL: https://issues.apache.org/jira/browse/CALCITE-3415
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.21.0
>Reporter: Pranay Parmar
>Assignee: Danny Chen
>Priority: Minor
>
> REGEXP_SUBSTR error :
> {code:java}
> No match found for function signature REGEXP_SUBSTR(, ,
> [, , ]){code}
>
> Example query:
> {code:sql}
> SELECT REGEXP_SUBSTR('chocolate Chip cookies', 'c+.{2}', 1, product_id, 'i')
> FROM public.account{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (CALCITE-3415) Supports REGEXP_SUBSTR function
[
https://issues.apache.org/jira/browse/CALCITE-3415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen reassigned CALCITE-3415:
---
Assignee: Danny Chen
> Supports REGEXP_SUBSTR function
> ---
>
> Key: CALCITE-3415
> URL: https://issues.apache.org/jira/browse/CALCITE-3415
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.21.0
>Reporter: Pranay Parmar
>Assignee: Danny Chen
>Priority: Minor
>
> REGEXP_SUBSTR error :
> {code:java}
> No match found for function signature REGEXP_SUBSTR(, ,
> [, , ]){code}
>
> Example query:
> {code:sql}
> SELECT REGEXP_SUBSTR('chocolate Chip cookies', 'c+.{2}', 1, product_id, 'i')
> FROM public.account{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3414) [CALCITE-3414] In calcite-core, use RexToLixTranslator.convert for type conversion code generation uniformly
[
https://issues.apache.org/jira/browse/CALCITE-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen updated CALCITE-3414:
Summary: [CALCITE-3414] In calcite-core, use RexToLixTranslator.convert for
type conversion code generation uniformly (was: Unify Expression'type cast and
conversion as a robust one)
> [CALCITE-3414] In calcite-core, use RexToLixTranslator.convert for type
> conversion code generation uniformly
>
>
> Key: CALCITE-3414
> URL: https://issues.apache.org/jira/browse/CALCITE-3414
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Attachments: RexToLixTranslator.png, TypeConversion.txt, Types.png
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Current now, there are two functions in calcite that can be used to
> cast/convert Expression to a specific Type.
> *_Types.castIfNecessary_* and _*RexToLixTranslator.convert*_.
> We make a deep investigation on their implementations and demonstrate them as
> below.
> {color:#ff}
> !RexToLixTranslator.png!{color}
> {color:#ff}
> *RexToLixTranslator.convert*{color}
> !Types.png!
> {color:#ff}
> *Types.castIfNecessary*{color}
> It can be seen that:
> (1) They have a lot of overlaps;
> (2) *_RexToLixTranslator.cast_* can cover more cases with tools like
> _SqlFunctions_ and etc.
> (3) Both of them have limitations and may generate incorrect code, which is
> listed in attachment(TypeConversion.txt).
> Multiple choices usually bring confusion to developers and resulting to the
> misuse of them.
> For example, CALCITE-3245 exposes that Types.castIfNecessary cannot cast the
> Expression to BigDecimal.class.
> Fixing the issue in *_Types.castIfNecessary_* directly seems to be not a
> good idea.
> On one hand, it is not convenient to call _SqlFunctions_ in linq4j. One the
> other hand, it will brings duplicate with _*RexToLixTranslator.cast*_.
> However, due to some unique logic in _*Types.castIfNecessary*_, we cannot
> replace it as _*RexToLixTranslator.cast*_ neither.
> Therefore, it is a good idea to integrate implementations into
> RexToLixTranslator.cast.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3414) In calcite-core, use RexToLixTranslator.convert for type conversion code generation uniformly
[
https://issues.apache.org/jira/browse/CALCITE-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen updated CALCITE-3414:
Summary: In calcite-core, use RexToLixTranslator.convert for type
conversion code generation uniformly (was: [CALCITE-3414] In calcite-core, use
RexToLixTranslator.convert for type conversion code generation uniformly)
> In calcite-core, use RexToLixTranslator.convert for type conversion code
> generation uniformly
> -
>
> Key: CALCITE-3414
> URL: https://issues.apache.org/jira/browse/CALCITE-3414
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Attachments: RexToLixTranslator.png, TypeConversion.txt, Types.png
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Current now, there are two functions in calcite that can be used to
> cast/convert Expression to a specific Type.
> *_Types.castIfNecessary_* and _*RexToLixTranslator.convert*_.
> We make a deep investigation on their implementations and demonstrate them as
> below.
> {color:#ff}
> !RexToLixTranslator.png!{color}
> {color:#ff}
> *RexToLixTranslator.convert*{color}
> !Types.png!
> {color:#ff}
> *Types.castIfNecessary*{color}
> It can be seen that:
> (1) They have a lot of overlaps;
> (2) *_RexToLixTranslator.cast_* can cover more cases with tools like
> _SqlFunctions_ and etc.
> (3) Both of them have limitations and may generate incorrect code, which is
> listed in attachment(TypeConversion.txt).
> Multiple choices usually bring confusion to developers and resulting to the
> misuse of them.
> For example, CALCITE-3245 exposes that Types.castIfNecessary cannot cast the
> Expression to BigDecimal.class.
> Fixing the issue in *_Types.castIfNecessary_* directly seems to be not a
> good idea.
> On one hand, it is not convenient to call _SqlFunctions_ in linq4j. One the
> other hand, it will brings duplicate with _*RexToLixTranslator.cast*_.
> However, due to some unique logic in _*Types.castIfNecessary*_, we cannot
> replace it as _*RexToLixTranslator.cast*_ neither.
> Therefore, it is a good idea to integrate implementations into
> RexToLixTranslator.cast.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3415) Supports REGEXP_SUBSTR function
[
https://issues.apache.org/jira/browse/CALCITE-3415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955728#comment-16955728
]
Pranay Parmar commented on CALCITE-3415:
[~danny0405] Yes I am still working on it. PR is ready just some testing.
Please don't let anyone else pick this up as I've already worked on it.
Thanks
> Supports REGEXP_SUBSTR function
> ---
>
> Key: CALCITE-3415
> URL: https://issues.apache.org/jira/browse/CALCITE-3415
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.21.0
>Reporter: Pranay Parmar
>Priority: Minor
>
> REGEXP_SUBSTR error :
> {code:java}
> No match found for function signature REGEXP_SUBSTR(, ,
> [, , ]){code}
>
> Example query:
> {code:sql}
> SELECT REGEXP_SUBSTR('chocolate Chip cookies', 'c+.{2}', 1, product_id, 'i')
> FROM public.account{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3428) Refine RelMdColumnUniqueness for Filter by considering constant columns
[
https://issues.apache.org/jira/browse/CALCITE-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955719#comment-16955719
]
jin xing commented on CALCITE-3428:
---
Thanks a lot [~julianhyde] ~
I will refine the PR by your comment !
> Refine RelMdColumnUniqueness for Filter by considering constant columns
> ---
>
> Key: CALCITE-3428
> URL: https://issues.apache.org/jira/browse/CALCITE-3428
> Project: Calcite
> Issue Type: Improvement
>Reporter: jin xing
>Assignee: jin xing
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> AggregateRemoveRule fails to remove the top Aggregate for below SQL
> {code:java}
> select mgr, sum(sum_sal)
> from
> (select mgr, deptno, sum(sal) sum_sal
> from sales.emp
> group by mgr, deptno)
> where deptno=100
> group by mgr
> {code}
> The reason is that RelMdColumnUniqueness doesn't take the filtering condition
> into consideration when checking uniqueness of columns.
> This PR proposes to refine RelMdColumnUniqueness for Filter, thus to
> strengthen AggregateRemoveRule.
> Resolving this Jira will help a lot for CALCITE-3334 by removing the
> redundant compensation Aggregate when doing materialization matching
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3429) Refinement for implementation of sql type factory
[ https://issues.apache.org/jira/browse/CALCITE-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang Yanlin updated CALCITE-3429: - Description: 1. Add map info from Java Map class to SqlTypeName.Map 2. SqlTypeName.Map is not a basic type, should not be used in factory method *createSqlType*, just like SqlTypeName.Array was: 1. The SqlTypeName of data type created with subclass of Map should be SqlTypename.Map, currently not. 2. The SqlTypeName of data type created with subclass of List or Array should be SqlTypename.Array, currently not. 3. SqlTypeName.Map is not a basic type, should not be used in factory method *createSqlType*, just like SqlTypeName.Array > Refinement for implementation of sql type factory > - > > Key: CALCITE-3429 > URL: https://issues.apache.org/jira/browse/CALCITE-3429 > Project: Calcite > Issue Type: Improvement >Reporter: Wang Yanlin >Assignee: Wang Yanlin >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > 1. Add map info from Java Map class to SqlTypeName.Map > 2. SqlTypeName.Map is not a basic type, should not be used in factory method > *createSqlType*, just like SqlTypeName.Array -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3415) Supports REGEXP_SUBSTR function
[
https://issues.apache.org/jira/browse/CALCITE-3415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen updated CALCITE-3415:
Summary: Supports REGEXP_SUBSTR function (was: Cannot parse REGEXP_SUBSTR
in BigQuery)
> Supports REGEXP_SUBSTR function
> ---
>
> Key: CALCITE-3415
> URL: https://issues.apache.org/jira/browse/CALCITE-3415
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.21.0
>Reporter: Pranay Parmar
>Priority: Minor
>
> REGEXP_SUBSTR error :
> {code:java}
> No match found for function signature REGEXP_SUBSTR(, ,
> [, , ]){code}
>
> Example query:
> {code:sql}
> SELECT REGEXP_SUBSTR('chocolate Chip cookies', 'c+.{2}', 1, product_id, 'i')
> FROM public.account{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3431) SemiJoinRule doesn't work when right-hand side is not Aggregate but distinct on join keys
[
https://issues.apache.org/jira/browse/CALCITE-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955705#comment-16955705
]
jin xing commented on CALCITE-3431:
---
Thanks for comment [~julianhyde]
I updated the Jira title to make it more clear ~
I created this Jira because semi-join is not created for the simple sql in the
Jira description.
Currently SemiJoinRule is mainly used to create semi-join from
sub-query(IN/EXISTS clause). As the discussion on
[https://github.com/apache/calcite/pull/1469], rather than refine SemiJoinRule,
I would prefer to make a change in SqlToRelConverter and have a direct path
from IN/EXISTS to semi- or anti-semi Join
> SemiJoinRule doesn't work when right-hand side is not Aggregate but distinct
> on join keys
> -
>
> Key: CALCITE-3431
> URL: https://issues.apache.org/jira/browse/CALCITE-3431
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: jin xing
>Assignee: jin xing
>Priority: Major
>
> Calcite generate semi-join by SemiJoinRule, which only works on pattern of
> Join(RelNode, Aggregate).
> Take below sql as an example
> {code:java}
> select * from dept
> where exists (
> select * from emp
> where emp.empno = dept.deptno)
> {code}
> If empno is a key column of emp, thus the original plan is as below
> {code:java}
> LogicalProject(DEPTNO=[$0], NAME=[$1])
> LogicalJoin(condition=[=($0, $2)], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalProject(EMPNO=[$0])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> Thus SemiJoinRule cannot support such a pattern
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3431) SemiJoinRule doesn't work when right-hand side is not Aggregate but distinct on join keys
[
https://issues.apache.org/jira/browse/CALCITE-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jin xing updated CALCITE-3431:
--
Summary: SemiJoinRule doesn't work when right-hand side is not Aggregate
but distinct on join keys (was: SemiJoinRule doesn't work when right-hand side
is natural distinct on join keys)
> SemiJoinRule doesn't work when right-hand side is not Aggregate but distinct
> on join keys
> -
>
> Key: CALCITE-3431
> URL: https://issues.apache.org/jira/browse/CALCITE-3431
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: jin xing
>Assignee: jin xing
>Priority: Major
>
> Calcite generate semi-join by SemiJoinRule, which only works on pattern of
> Join(RelNode, Aggregate).
> Take below sql as an example
> {code:java}
> select * from dept
> where exists (
> select * from emp
> where emp.empno = dept.deptno)
> {code}
> If empno is a key column of emp, thus the original plan is as below
> {code:java}
> LogicalProject(DEPTNO=[$0], NAME=[$1])
> LogicalJoin(condition=[=($0, $2)], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalProject(EMPNO=[$0])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> Thus SemiJoinRule cannot support such a pattern
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3395) add BuiltinMethod for Substring(String, int)
[ https://issues.apache.org/jira/browse/CALCITE-3395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955701#comment-16955701 ] Youjun Yuan commented on CALCITE-3395: -- Hi Julian, For "addMonths", "overlay", we do have different entries for different versions, so adding another entry for "substring" would be a nature fix. We are leveraging calcite to generate golang code, which does not support method overload, hence it's a PROBLEM for us. Though I can walk around this, I thought fixing it in calcite would make other's life a little bit easier (in similar scenario). I'd like to contribute on this, once we agree it's the right way to go. > add BuiltinMethod for Substring(String, int) > > > Key: CALCITE-3395 > URL: https://issues.apache.org/jira/browse/CALCITE-3395 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Youjun Yuan >Priority: Minor > Original Estimate: 4h > Remaining Estimate: 4h > > substring function has two versoins: > 1, Substring(String, int, int) > 2, Substring(String, int) > currently in BuiltinMethod.java, only the first one is defined. Need to > define the second one as well, so that we can use both of them. > Apache Flink(FunctionGenerator.scala), reference the BuiltinMethod, since > Calcite only defines 1, not 2, Flink always resolve SUBSTRING to > Substring(String, int, int) even if there is only 2 parameters. This problem > happens to be covered by method overloading of java, but it's still a > potential problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (CALCITE-3432) RexBuilder construct method with the default of RelDataTypeFactory.
[ https://issues.apache.org/jira/browse/CALCITE-3432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed CALCITE-3432. --- Resolution: Won't Do > RexBuilder construct method with the default of RelDataTypeFactory. > --- > > Key: CALCITE-3432 > URL: https://issues.apache.org/jira/browse/CALCITE-3432 > Project: Calcite > Issue Type: Wish >Reporter: xzh_dz >Priority: Minor > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > RexBuilder construct method with the default of RelDataTypeFactory. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3414) Unify Expression'type cast and conversion as a robust one
[
https://issues.apache.org/jira/browse/CALCITE-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955692#comment-16955692
]
Danny Chen commented on CALCITE-3414:
-
Reviewing now ~
> Unify Expression'type cast and conversion as a robust one
> -
>
> Key: CALCITE-3414
> URL: https://issues.apache.org/jira/browse/CALCITE-3414
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Attachments: RexToLixTranslator.png, TypeConversion.txt, Types.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Current now, there are two functions in calcite that can be used to
> cast/convert Expression to a specific Type.
> *_Types.castIfNecessary_* and _*RexToLixTranslator.convert*_.
> We make a deep investigation on their implementations and demonstrate them as
> below.
> {color:#ff}
> !RexToLixTranslator.png!{color}
> {color:#ff}
> *RexToLixTranslator.convert*{color}
> !Types.png!
> {color:#ff}
> *Types.castIfNecessary*{color}
> It can be seen that:
> (1) They have a lot of overlaps;
> (2) *_RexToLixTranslator.cast_* can cover more cases with tools like
> _SqlFunctions_ and etc.
> (3) Both of them have limitations and may generate incorrect code, which is
> listed in attachment(TypeConversion.txt).
> Multiple choices usually bring confusion to developers and resulting to the
> misuse of them.
> For example, CALCITE-3245 exposes that Types.castIfNecessary cannot cast the
> Expression to BigDecimal.class.
> Fixing the issue in *_Types.castIfNecessary_* directly seems to be not a
> good idea.
> On one hand, it is not convenient to call _SqlFunctions_ in linq4j. One the
> other hand, it will brings duplicate with _*RexToLixTranslator.cast*_.
> However, due to some unique logic in _*Types.castIfNecessary*_, we cannot
> replace it as _*RexToLixTranslator.cast*_ neither.
> Therefore, it is a good idea to integrate implementations into
> RexToLixTranslator.cast.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3426) Complement validConstant type in RexLiteral
[ https://issues.apache.org/jira/browse/CALCITE-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated CALCITE-3426: Summary: Complement validConstant type in RexLiteral (was: compensate validConstant type in RexLiteral.) > Complement validConstant type in RexLiteral > --- > > Key: CALCITE-3426 > URL: https://issues.apache.org/jira/browse/CALCITE-3426 > Project: Calcite > Issue Type: Wish >Reporter: xzh_dz >Priority: Minor > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > compensate validConstant type in RexLiteral. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3426) compensate validConstant type in RexLiteral.
[
https://issues.apache.org/jira/browse/CALCITE-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955680#comment-16955680
]
Julian Hyde edited comment on CALCITE-3426 at 10/21/19 1:53 AM:
Not sure what "compensate" means. Can you improve the description.
As [~danny0405] notes, this method is currently only called in tests. If it
were called from {{RexLiteral}}'s constructor, or from {{valueMatchesType}}, it
would be more trustworthy. There would be some performance impact, but probably
OK.
Alternatively, write a test that calls both {{validConstant}} and
{{valueMatchesType}}, in both positive and negative tests, and check that they
always agree. Which is what the contract of {{validConstant}} promises, after
all.
was (Author: julianhyde):
Not sure what "compensate" means. Can you improve the description.
As [~danny0405] notes, this method is currently only called in tests. If it
were called from RexLiteral's constructor, or from valueMatchesType, it would
be more trustworthy.
> compensate validConstant type in RexLiteral.
>
>
> Key: CALCITE-3426
> URL: https://issues.apache.org/jira/browse/CALCITE-3426
> Project: Calcite
> Issue Type: Wish
>Reporter: xzh_dz
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> compensate validConstant type in RexLiteral.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3323) Handle arbitrary/unknown functions that have ordinary syntax
[ https://issues.apache.org/jira/browse/CALCITE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-3323: Labels: pull-request-available (was: ) > Handle arbitrary/unknown functions that have ordinary syntax > > > Key: CALCITE-3323 > URL: https://issues.apache.org/jira/browse/CALCITE-3323 > Project: Calcite > Issue Type: New Feature >Reporter: Ryan Fu >Priority: Major > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Add a strategy where if a function has ordinary function syntax and we don't > recognize it we assume that it can take any argument types and returns a > result of unknown type. > We will still need to change the parser to handle functions with non-standard > syntax (e.g. DATEADD). And it's a good idea to explicitly add commonly used > non-standard functions (e.g. MD5, CONCAT).” > [19/Sept/2019] Functions have to have ordinary function syntax [e.g. > foo(arg0, ...)]. Current implementation idea is to: > * Create a known/unknown flag for use within the validator > * Create a more permissive SqlUnresolvedFunction to be passed through > validator > ** Allow OperandTypes to be variadic -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3323) Handle arbitrary/unknown functions that have ordinary syntax
[ https://issues.apache.org/jira/browse/CALCITE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955683#comment-16955683 ] Julian Hyde commented on CALCITE-3323: -- I have removed the link to PR #1522; this issue is fixed, and that PR needs a new issue to be logged. It is only somewhat related to this case. > Handle arbitrary/unknown functions that have ordinary syntax > > > Key: CALCITE-3323 > URL: https://issues.apache.org/jira/browse/CALCITE-3323 > Project: Calcite > Issue Type: New Feature >Reporter: Ryan Fu >Priority: Major > Fix For: 1.22.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Add a strategy where if a function has ordinary function syntax and we don't > recognize it we assume that it can take any argument types and returns a > result of unknown type. > We will still need to change the parser to handle functions with non-standard > syntax (e.g. DATEADD). And it's a good idea to explicitly add commonly used > non-standard functions (e.g. MD5, CONCAT).” > [19/Sept/2019] Functions have to have ordinary function syntax [e.g. > foo(arg0, ...)]. Current implementation idea is to: > * Create a known/unknown flag for use within the validator > * Create a more permissive SqlUnresolvedFunction to be passed through > validator > ** Allow OperandTypes to be variadic -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3323) Handle arbitrary/unknown functions that have ordinary syntax
[ https://issues.apache.org/jira/browse/CALCITE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde updated CALCITE-3323: - Labels: (was: pull-request-available) > Handle arbitrary/unknown functions that have ordinary syntax > > > Key: CALCITE-3323 > URL: https://issues.apache.org/jira/browse/CALCITE-3323 > Project: Calcite > Issue Type: New Feature >Reporter: Ryan Fu >Priority: Major > Fix For: 1.22.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Add a strategy where if a function has ordinary function syntax and we don't > recognize it we assume that it can take any argument types and returns a > result of unknown type. > We will still need to change the parser to handle functions with non-standard > syntax (e.g. DATEADD). And it's a good idea to explicitly add commonly used > non-standard functions (e.g. MD5, CONCAT).” > [19/Sept/2019] Functions have to have ordinary function syntax [e.g. > foo(arg0, ...)]. Current implementation idea is to: > * Create a known/unknown flag for use within the validator > * Create a more permissive SqlUnresolvedFunction to be passed through > validator > ** Allow OperandTypes to be variadic -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3426) compensate validConstant type in RexLiteral.
[ https://issues.apache.org/jira/browse/CALCITE-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955680#comment-16955680 ] Julian Hyde commented on CALCITE-3426: -- Not sure what "compensate" means. Can you improve the description. As [~danny0405] notes, this method is currently only called in tests. If it were called from RexLiteral's constructor, or from valueMatchesType, it would be more trustworthy. > compensate validConstant type in RexLiteral. > > > Key: CALCITE-3426 > URL: https://issues.apache.org/jira/browse/CALCITE-3426 > Project: Calcite > Issue Type: Wish >Reporter: xzh_dz >Priority: Minor > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > compensate validConstant type in RexLiteral. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3432) RexBuilder construct method with the default of RelDataTypeFactory.
[
https://issues.apache.org/jira/browse/CALCITE-3432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955676#comment-16955676
]
Julian Hyde commented on CALCITE-3432:
--
I don't think this is a useful change. People might accidentally create a
RexBuilder with a different type factory than is used elsewhere in their code.
There's probably some code in {{Planner}} or {{Frameworks}} that creates a
{{RelDataTypeFactory}}, {{RexBuilder}} and other objects that people might
need, such as {{RelBuilder}}.
Or you can add it to your own utility class. I'd recommend that you add javadoc
explaining what it is doing.
> RexBuilder construct method with the default of RelDataTypeFactory.
> ---
>
> Key: CALCITE-3432
> URL: https://issues.apache.org/jira/browse/CALCITE-3432
> Project: Calcite
> Issue Type: Wish
>Reporter: xzh_dz
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> RexBuilder construct method with the default of RelDataTypeFactory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3431) SemiJoinRule doesn't work when right-hand side is natural distinct on join keys
[
https://issues.apache.org/jira/browse/CALCITE-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955671#comment-16955671
]
Julian Hyde commented on CALCITE-3431:
--
The word "natural" in the description confused me. You're not talking about
"natural join" here.
Yes it makes sense to have a variant of SemiJoinRule that looks for uniqueness
not an Aggregate per se. However, you will need to be careful that the rule
doesn't cycle.
Also make sure that people can opt out of this functionality. Based on
discussions on the mail list, not everyone considers it useful to merely
convert join -> semiJoin if no other RelNodes are removed.
> SemiJoinRule doesn't work when right-hand side is natural distinct on join
> keys
> ---
>
> Key: CALCITE-3431
> URL: https://issues.apache.org/jira/browse/CALCITE-3431
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: jin xing
>Assignee: jin xing
>Priority: Major
>
> Calcite generate semi-join by SemiJoinRule, which only works on pattern of
> Join(RelNode, Aggregate).
> Take below sql as an example
> {code:java}
> select * from dept
> where exists (
> select * from emp
> where emp.empno = dept.deptno)
> {code}
> If empno is a key column of emp, thus the original plan is as below
> {code:java}
> LogicalProject(DEPTNO=[$0], NAME=[$1])
> LogicalJoin(condition=[=($0, $2)], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalProject(EMPNO=[$0])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> Thus SemiJoinRule cannot support such a pattern
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3428) Refine RelMdColumnUniqueness for Filter by considering constant columns
[
https://issues.apache.org/jira/browse/CALCITE-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955666#comment-16955666
]
Julian Hyde commented on CALCITE-3428:
--
This is a useful change; I'm glad you're doing it.
Did you consider driving from {{RelMdPredicates}}? {{Filter}} creates
predicates, but so do many other kinds of {{RelNode}}, and moreover, predicates
are propagated. So if you use {{RelMdPredicates}} this rule will apply in more
cases.
Also, it will allow you to use {{RelOptPredicateList.constantMap}}, which will
save you the effort of parsing conditions to look for conditions of the form
'column = constant'.
{{testAggregateRemove7}} needs a javadoc comment to explain its purpose.
> Refine RelMdColumnUniqueness for Filter by considering constant columns
> ---
>
> Key: CALCITE-3428
> URL: https://issues.apache.org/jira/browse/CALCITE-3428
> Project: Calcite
> Issue Type: Improvement
>Reporter: jin xing
>Assignee: jin xing
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> AggregateRemoveRule fails to remove the top Aggregate for below SQL
> {code:java}
> select mgr, sum(sum_sal)
> from
> (select mgr, deptno, sum(sal) sum_sal
> from sales.emp
> group by mgr, deptno)
> where deptno=100
> group by mgr
> {code}
> The reason is that RelMdColumnUniqueness doesn't take the filtering condition
> into consideration when checking uniqueness of columns.
> This PR proposes to refine RelMdColumnUniqueness for Filter, thus to
> strengthen AggregateRemoveRule.
> Resolving this Jira will help a lot for CALCITE-3334 by removing the
> redundant compensation Aggregate when doing materialization matching
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3432) RexBuilder construct method with the default of RelDataTypeFactory.
[ https://issues.apache.org/jira/browse/CALCITE-3432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-3432: Labels: pull-request-available (was: ) > RexBuilder construct method with the default of RelDataTypeFactory. > --- > > Key: CALCITE-3432 > URL: https://issues.apache.org/jira/browse/CALCITE-3432 > Project: Calcite > Issue Type: Wish >Reporter: xzh_dz >Priority: Minor > Labels: pull-request-available > > RexBuilder construct method with the default of RelDataTypeFactory. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3432) RexBuilder construct method with the default of RelDataTypeFactory.
xzh_dz created CALCITE-3432: --- Summary: RexBuilder construct method with the default of RelDataTypeFactory. Key: CALCITE-3432 URL: https://issues.apache.org/jira/browse/CALCITE-3432 Project: Calcite Issue Type: Wish Reporter: xzh_dz RexBuilder construct method with the default of RelDataTypeFactory. -- This message was sent by Atlassian Jira (v8.3.4#803005)
