[jira] [Commented] (CALCITE-883) Support RESPECT NULLS, IGNORE NULLS option for LEAD, LAG, FIRST_VALUE, LAST_VALUE window functions
[ https://issues.apache.org/jira/browse/CALCITE-883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799540#comment-16799540 ] Chunwei Lei commented on CALCITE-883: - [~zhztheplayer] Thank you for your patient explanation. It may help to decide how to do it if someone else can join the discussion. [~julianhyde] What do you think of it? > Support RESPECT NULLS, IGNORE NULLS option for LEAD, LAG, FIRST_VALUE, > LAST_VALUE window functions > -- > > Key: CALCITE-883 > URL: https://issues.apache.org/jira/browse/CALCITE-883 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Sean Hsuan-Yi Chu >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > This issue was brought up in Drill community: > (https://issues.apache.org/jira/browse/DRILL-3597) > For convenience, I copied and pasted the reporter Khurram Faraaz's > description here: > "The SQL standard defines a RESPECT NULLS or IGNORE NULLS option for lead, > lag, first_value, and last_value window functions." > In Calcite, we should have been able to recognized this option. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2900) RewriteRexShuttle generates wrong type during flatten for structured type when visitInputRef
[ https://issues.apache.org/jira/browse/CALCITE-2900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799536#comment-16799536 ] Will Yu commented on CALCITE-2900: -- [~zabetak] Hey man, could you help assign this issue to me? And could you change me to a Calcite contributor? (Not sure whether there is such a process, just check) Thanks. > RewriteRexShuttle generates wrong type during flatten for structured type > when visitInputRef > > > Key: CALCITE-2900 > URL: https://issues.apache.org/jira/browse/CALCITE-2900 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.18.0 >Reporter: Will Yu >Priority: Blocker > Labels: pull-request-available > Time Spent: 3h 50m > Remaining Estimate: 0h > > When we leverage Calcite to parse a SQL and generates a RelNode tree on top > of a table with *structured* columns, Calcite throws an error about type > mismatch shown at bottom. > After a bit of debugging, the issue is suspected to happen in > RewriteRexShuttle.visitFieldAccess. When the input is RexInputRef, such as > $36.target_column, the newField type is calculated based on ONLY the index > 36, WITHOUT considering the offset of target_column. The result is new field > type is the type of the first sub-column of column $36, which causes the type > mismatch. > > Error and stacktrace: > java.lang.AssertionError: type mismatch: > ref: > INTEGER NOT NULL > input: > VARCHAR CHARACTER SET "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary" NOT NULL > at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31) > at org.apache.calcite.plan.RelOptUtil.eq(RelOptUtil.java:1832) > at org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:125) > at org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:57) > at org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:112) > at org.apache.calcite.rel.core.Project.isValid(Project.java:186) > at org.apache.calcite.rel.core.Project.(Project.java:83) > at > org.apache.calcite.rel.logical.LogicalProject.(LogicalProject.java:62) > at > org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:112) > at > org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:100) > at > org.apache.calcite.rel.core.RelFactories$ProjectFactoryImpl.createProject(RelFactories.java:140) > at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1248) > at org.apache.calcite.tools.RelBuilder.projectNamed(RelBuilder.java:1304) > at > org.apache.calcite.sql2rel.RelStructuredTypeFlattener.rewriteRel(RelStructuredTypeFlattener.java:506) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.calcite.util.ReflectUtil.invokeVisitorInternal(ReflectUtil.java:257) > at org.apache.calcite.util.ReflectUtil.invokeVisitor(ReflectUtil.java:214) > at org.apache.calcite.util.ReflectUtil$1.invokeVisitor(ReflectUtil.java:464) > at > org.apache.calcite.sql2rel.RelStructuredTypeFlattener$RewriteRelVisitor.visit(RelStructuredTypeFlattener.java:759) > at org.apache.calcite.rel.SingleRel.childrenAccept(SingleRel.java:72) > at org.apache.calcite.rel.RelVisitor.visit(RelVisitor.java:44) > at > org.apache.calcite.sql2rel.RelStructuredTypeFlattener$RewriteRelVisitor.visit(RelStructuredTypeFlattener.java:754) > at > org.apache.calcite.sql2rel.RelStructuredTypeFlattener.rewrite(RelStructuredTypeFlattener.java:194) > at > org.apache.calcite.sql2rel.SqlToRelConverter.flattenTypes(SqlToRelConverter.java:466) > at org.apache.calcite.prepare.PlannerImpl.rel(PlannerImpl.java:236) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-883) Support RESPECT NULLS, IGNORE NULLS option for LEAD, LAG, FIRST_VALUE, LAST_VALUE window functions
[ https://issues.apache.org/jira/browse/CALCITE-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunwei Lei updated CALCITE-883: Attachment: (was: image-2019-03-23-13-03-47-184.png) > Support RESPECT NULLS, IGNORE NULLS option for LEAD, LAG, FIRST_VALUE, > LAST_VALUE window functions > -- > > Key: CALCITE-883 > URL: https://issues.apache.org/jira/browse/CALCITE-883 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Sean Hsuan-Yi Chu >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > This issue was brought up in Drill community: > (https://issues.apache.org/jira/browse/DRILL-3597) > For convenience, I copied and pasted the reporter Khurram Faraaz's > description here: > "The SQL standard defines a RESPECT NULLS or IGNORE NULLS option for lead, > lag, first_value, and last_value window functions." > In Calcite, we should have been able to recognized this option. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-883) Support RESPECT NULLS, IGNORE NULLS option for LEAD, LAG, FIRST_VALUE, LAST_VALUE window functions
[ https://issues.apache.org/jira/browse/CALCITE-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunwei Lei updated CALCITE-883: Attachment: image-2019-03-23-13-03-47-184.png > Support RESPECT NULLS, IGNORE NULLS option for LEAD, LAG, FIRST_VALUE, > LAST_VALUE window functions > -- > > Key: CALCITE-883 > URL: https://issues.apache.org/jira/browse/CALCITE-883 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Sean Hsuan-Yi Chu >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Attachments: image-2019-03-23-13-03-47-184.png > > Time Spent: 1h 20m > Remaining Estimate: 0h > > This issue was brought up in Drill community: > (https://issues.apache.org/jira/browse/DRILL-3597) > For convenience, I copied and pasted the reporter Khurram Faraaz's > description here: > "The SQL standard defines a RESPECT NULLS or IGNORE NULLS option for lead, > lag, first_value, and last_value window functions." > In Calcite, we should have been able to recognized this option. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2936) Simplify EXISTS or NOT EXISTS sub-query that has "GROUP BY ()"
[
https://issues.apache.org/jira/browse/CALCITE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799519#comment-16799519
]
Julian Hyde commented on CALCITE-2936:
--
{{MinRowCount}} is an estimate, but it is a safe lower bound. If
{{MinRowCount}} evaluates to 2 for a particular relational expression, then the
relational expression might return 2 rows or 200, but will never return 1 or 0
rows.
Similarly {{MaxRowCount}}.
They are intended for precisely these kinds of optimizations. I'm sorry that
the documentation doesn't make that crystal clear.
> Simplify EXISTS or NOT EXISTS sub-query that has "GROUP BY ()"
> --
>
> Key: CALCITE-2936
> URL: https://issues.apache.org/jira/browse/CALCITE-2936
> Project: Calcite
> Issue Type: New Feature
>Reporter: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> An EXISTS or NOT EXISTS sub-query whose inner child is an aggregate with no
> grouping columns should be simplified to a Boolean constant.
> Example:
> {code:java}
> exists(select sum(i) from X) --> true
> not exists(select sum(i) from X) --> false
> {code}
> Repro:
> {code:java}
> @Test public void testExistentialSubquery() {
> final String sql = "SELECT e1.empno\n"
> + "FROM emp e1 where exists\n"
> + "(select avg(sal) from emp e2 where e1.empno = e2.empno )";
> sql(sql).decorrelate(true).ok();
> }
> {code}
> We got plan:
> {code:java}
> LogicalProject(EMPNO=[$0])
> LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
> SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], EMPNO0=[CAST($9):INTEGER],
> $f1=[CAST($10):BOOLEAN])
> LogicalJoin(condition=[=($0, $9)], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalAggregate(group=[{0}], agg#0=[MIN($1)])
> LogicalProject(EMPNO=[$0], $f0=[true])
> LogicalAggregate(group=[{0}], EXPR$0=[AVG($1)])
> LogicalProject(EMPNO=[$0], SAL=[$5])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> The preferred plan should be:
> {code:java}
> LogicalProject(EMPNO=[$0])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2946) RelBuilder wrongly skips creation of Aggregate that prunes columns if input produces one row at most
[
https://issues.apache.org/jira/browse/CALCITE-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-2946:
Labels: pull-request-available (was: )
> RelBuilder wrongly skips creation of Aggregate that prunes columns if input
> produces one row at most
>
>
> Key: CALCITE-2946
> URL: https://issues.apache.org/jira/browse/CALCITE-2946
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Prasanth Jayachandran
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
>
> Calcite throws the following error when UDFs return null.
> "current_authorizer()" for example can return null if authorizer is disabled.
> {code:java}
> org.apache.hive.service.cli.HiveSQLException: Error running query:
> java.lang.AssertionError: Cannot add expression of different type to set:
> set type is RecordType(CHAR(7) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" NOT NULL $f0, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f1, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f2,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f3, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f4, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f5, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f6,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f7, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f8, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f9, CHAR(2) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" NOT NULL $f10,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f11) NOT NULL
> expression type is RecordType(CHAR(7) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" NOT NULL $f0, CHAR(7) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" NOT NULL $f1, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f2,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f3, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f4, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f5, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f6,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f7, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f8, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f9, CHAR(2) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" NOT NULL $f10,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f11) NOT NULL
> set is rel#784:HiveAggregate.HIVE.[](input=HepRelVertex#783,group={0, 1, 2,
> 3, 4, 5, 6, 7, 8, 9, 10, 11})
> expression is HiveProject#829
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:210)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:322)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:342)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.AssertionError: Cannot add expression of different type
> to set:
> set type is RecordType(CHAR(7) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" NOT NULL $f0, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f1, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f2,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f3, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE
[jira] [Updated] (CALCITE-2946) RelBuilder wrongly skips creation of Aggregate that prunes columns if input produces one row at most
[
https://issues.apache.org/jira/browse/CALCITE-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-2946:
-
Summary: RelBuilder wrongly skips creation of Aggregate that prunes columns
if input produces one row at most (was: Calcite assertion error when UDF
returns null)
> RelBuilder wrongly skips creation of Aggregate that prunes columns if input
> produces one row at most
>
>
> Key: CALCITE-2946
> URL: https://issues.apache.org/jira/browse/CALCITE-2946
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Prasanth Jayachandran
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> Calcite throws the following error when UDFs return null.
> "current_authorizer()" for example can return null if authorizer is disabled.
> {code:java}
> org.apache.hive.service.cli.HiveSQLException: Error running query:
> java.lang.AssertionError: Cannot add expression of different type to set:
> set type is RecordType(CHAR(7) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" NOT NULL $f0, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f1, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f2,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f3, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f4, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f5, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f6,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f7, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f8, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f9, CHAR(2) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" NOT NULL $f10,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f11) NOT NULL
> expression type is RecordType(CHAR(7) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" NOT NULL $f0, CHAR(7) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" NOT NULL $f1, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f2,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f3, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f4, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f5, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f6,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f7, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f8, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f9, CHAR(2) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" NOT NULL $f10,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f11) NOT NULL
> set is rel#784:HiveAggregate.HIVE.[](input=HepRelVertex#783,group={0, 1, 2,
> 3, 4, 5, 6, 7, 8, 9, 10, 11})
> expression is HiveProject#829
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:210)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:322)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:342)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.AssertionError: Cannot add expression of different type
> to set:
> set type is RecordType(CHAR(7) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" NOT NULL $f0, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f1, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f2,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
[jira] [Commented] (CALCITE-2946) Calcite assertion error when UDF returns null
[
https://issues.apache.org/jira/browse/CALCITE-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799453#comment-16799453
]
Jesus Camacho Rodriguez commented on CALCITE-2946:
--
This is very similar to CALCITE-2192. An Aggregate may not be created if
expression is known to produce one row at most:
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/tools/RelBuilder.java#L1434
However, the Aggregate may prune columns in the expression. Fix is similar, a
Project should be created. I still need to find a simple repro for Calcite.
> Calcite assertion error when UDF returns null
> -
>
> Key: CALCITE-2946
> URL: https://issues.apache.org/jira/browse/CALCITE-2946
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Prasanth Jayachandran
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> Calcite throws the following error when UDFs return null.
> "current_authorizer()" for example can return null if authorizer is disabled.
> {code:java}
> org.apache.hive.service.cli.HiveSQLException: Error running query:
> java.lang.AssertionError: Cannot add expression of different type to set:
> set type is RecordType(CHAR(7) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" NOT NULL $f0, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f1, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f2,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f3, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f4, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f5, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f6,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f7, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f8, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f9, CHAR(2) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" NOT NULL $f10,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f11) NOT NULL
> expression type is RecordType(CHAR(7) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" NOT NULL $f0, CHAR(7) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" NOT NULL $f1, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f2,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f3, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f4, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f5, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f6,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f7, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f8, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f9, CHAR(2) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" NOT NULL $f10,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f11) NOT NULL
> set is rel#784:HiveAggregate.HIVE.[](input=HepRelVertex#783,group={0, 1, 2,
> 3, 4, 5, 6, 7, 8, 9, 10, 11})
> expression is HiveProject#829
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:210)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:322)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:342)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.AssertionError: Cannot add expression of different type
> to set:
> set type is RecordType(CHAR(7) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" NOT NULL $f0, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary"
[jira] [Moved] (CALCITE-2946) Calcite assertion error when UDF returns null
[
https://issues.apache.org/jira/browse/CALCITE-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez moved HIVE-21495 to CALCITE-2946:
-
Affects Version/s: (was: 4.0.0)
Target Version/s: (was: 4.0.0)
Component/s: (was: Logical Optimizer)
core
Workflow: jira (was: no-reopen-closed, patch-avail)
Key: CALCITE-2946 (was: HIVE-21495)
Project: Calcite (was: Hive)
> Calcite assertion error when UDF returns null
> -
>
> Key: CALCITE-2946
> URL: https://issues.apache.org/jira/browse/CALCITE-2946
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Prasanth Jayachandran
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> Calcite throws the following error when UDFs return null.
> "current_authorizer()" for example can return null if authorizer is disabled.
> {code:java}
> org.apache.hive.service.cli.HiveSQLException: Error running query:
> java.lang.AssertionError: Cannot add expression of different type to set:
> set type is RecordType(CHAR(7) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" NOT NULL $f0, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f1, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f2,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f3, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f4, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f5, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f6,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f7, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f8, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f9, CHAR(2) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" NOT NULL $f10,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f11) NOT NULL
> expression type is RecordType(CHAR(7) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" NOT NULL $f0, CHAR(7) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" NOT NULL $f1, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f2,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f3, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f4, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f5, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f6,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f7, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" $f8, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f9, CHAR(2) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" NOT NULL $f10,
> VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" $f11) NOT NULL
> set is rel#784:HiveAggregate.HIVE.[](input=HepRelVertex#783,group={0, 1, 2,
> 3, 4, 5, 6, 7, 8, 9, 10, 11})
> expression is HiveProject#829
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:210)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:322)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:342)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.AssertionError: Cannot add expression of different type
> to set:
> set type is RecordType(CHAR(7) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" NOT NULL $f0, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f1, VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE" COLLATE
[jira] [Updated] (CALCITE-2913) add a KafkaAdapter for Stream
[ https://issues.apache.org/jira/browse/CALCITE-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-2913: Labels: pull-request-available (was: ) > add a KafkaAdapter for Stream > - > > Key: CALCITE-2913 > URL: https://issues.apache.org/jira/browse/CALCITE-2913 > Project: Calcite > Issue Type: New Feature > Components: core >Reporter: Xu Mingmin >Assignee: James Xu >Priority: Major > Labels: pull-request-available > > Add a new adapter which exposes a Kafka topic into a Stream table, we've one > version in our env and would like to share it if not duplicated. > Hint, we are actually extending it as a batch table, limiting to Stream table > sounds more straight-forward to me as 1st step. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2944) Lack of test for Aggregate.indicator
[ https://issues.apache.org/jira/browse/CALCITE-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799272#comment-16799272 ] Haisheng Yuan commented on CALCITE-2944: Yeah, the field and constructor using indicator should be marked deprecated explicitly, by annotation. Tests should be there to guarantee its correctness, though. > Lack of test for Aggregate.indicator > > > Key: CALCITE-2944 > URL: https://issues.apache.org/jira/browse/CALCITE-2944 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Priority: Major > Attachments: image-2019-03-21-18-19-58-933.png > > > Although Aggregate.indicator is discouraged to use, but no test cases > guarantee indicator can work correctly. I tried to set indicator to always > false in Aggregate constructor, it passed all the tests. > !image-2019-03-21-18-19-58-933.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2936) Simplify EXISTS or NOT EXISTS sub-query that has "GROUP BY ()"
[
https://issues.apache.org/jira/browse/CALCITE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799268#comment-16799268
]
Haisheng Yuan commented on CALCITE-2936:
[~danny0405] Let's continue discussion here. I understand that the description
in the code concerns you, but I think the comments are misleading, the word
should be determine, not estimate. The minRowCount and maxRowCount are provided
to help determine whether we can do further optimization, like aggregate / sort
removal, existential check, not for cardinality and cost estimation. I don't
how much value it will provide if it is an estimate value. Should we update the
comments If I don't misunderstand the intention?
> Simplify EXISTS or NOT EXISTS sub-query that has "GROUP BY ()"
> --
>
> Key: CALCITE-2936
> URL: https://issues.apache.org/jira/browse/CALCITE-2936
> Project: Calcite
> Issue Type: New Feature
>Reporter: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> An EXISTS or NOT EXISTS sub-query whose inner child is an aggregate with no
> grouping columns should be simplified to a Boolean constant.
> Example:
> {code:java}
> exists(select sum(i) from X) --> true
> not exists(select sum(i) from X) --> false
> {code}
> Repro:
> {code:java}
> @Test public void testExistentialSubquery() {
> final String sql = "SELECT e1.empno\n"
> + "FROM emp e1 where exists\n"
> + "(select avg(sal) from emp e2 where e1.empno = e2.empno )";
> sql(sql).decorrelate(true).ok();
> }
> {code}
> We got plan:
> {code:java}
> LogicalProject(EMPNO=[$0])
> LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
> SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], EMPNO0=[CAST($9):INTEGER],
> $f1=[CAST($10):BOOLEAN])
> LogicalJoin(condition=[=($0, $9)], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalAggregate(group=[{0}], agg#0=[MIN($1)])
> LogicalProject(EMPNO=[$0], $f0=[true])
> LogicalAggregate(group=[{0}], EXPR$0=[AVG($1)])
> LogicalProject(EMPNO=[$0], SAL=[$5])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> The preferred plan should be:
> {code:java}
> LogicalProject(EMPNO=[$0])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2936) Simplify EXISTS or NOT EXISTS sub-query that has "GROUP BY ()"
[
https://issues.apache.org/jira/browse/CALCITE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799254#comment-16799254
]
Haisheng Yuan commented on CALCITE-2936:
In order to remove semijoin, we have to determine that for every tuple from
outer rel, there must be one and only one matching tuple in the inner rel. I
have no clue for the example query in this JIRA.
> Simplify EXISTS or NOT EXISTS sub-query that has "GROUP BY ()"
> --
>
> Key: CALCITE-2936
> URL: https://issues.apache.org/jira/browse/CALCITE-2936
> Project: Calcite
> Issue Type: New Feature
>Reporter: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> An EXISTS or NOT EXISTS sub-query whose inner child is an aggregate with no
> grouping columns should be simplified to a Boolean constant.
> Example:
> {code:java}
> exists(select sum(i) from X) --> true
> not exists(select sum(i) from X) --> false
> {code}
> Repro:
> {code:java}
> @Test public void testExistentialSubquery() {
> final String sql = "SELECT e1.empno\n"
> + "FROM emp e1 where exists\n"
> + "(select avg(sal) from emp e2 where e1.empno = e2.empno )";
> sql(sql).decorrelate(true).ok();
> }
> {code}
> We got plan:
> {code:java}
> LogicalProject(EMPNO=[$0])
> LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
> SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], EMPNO0=[CAST($9):INTEGER],
> $f1=[CAST($10):BOOLEAN])
> LogicalJoin(condition=[=($0, $9)], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalAggregate(group=[{0}], agg#0=[MIN($1)])
> LogicalProject(EMPNO=[$0], $f0=[true])
> LogicalAggregate(group=[{0}], EXPR$0=[AVG($1)])
> LogicalProject(EMPNO=[$0], SAL=[$5])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> The preferred plan should be:
> {code:java}
> LogicalProject(EMPNO=[$0])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2941) EnumerableLimitRule on Sort with no collation creates EnumerableLimit with wrong traitSet and cluster
[ https://issues.apache.org/jira/browse/CALCITE-2941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799216#comment-16799216 ] Ruben Quesada Lopez commented on CALCITE-2941: -- Provided unit test > EnumerableLimitRule on Sort with no collation creates EnumerableLimit with > wrong traitSet and cluster > - > > Key: CALCITE-2941 > URL: https://issues.apache.org/jira/browse/CALCITE-2941 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.18.0 >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 10m > Remaining Estimate: 0h > > EnumerableLimitRule _"converst a Sort that has offset or fetch set to an > EnumerableLimit on top of a 'pure' Sort that has no offset or fetch"_. This > is the "normal" scenario, and there is no issue with that. > However, there is another scenario, where the EnumerableLimitRule is applied > on an "empty" Sort (with no field collations) with offset / fetch. In that > case, the EnumerableLimit's input will not be the Sort (because it will > disappear, since it was empty), but the Sort's input. The problem comes in > this case, because the EnumerableLimit will be created with the Sort's > traitSet and cluster; instead of the Sort's input (which is the actual > EnumerableLimit input) traitSet and cluster. > Probably the bug can be easily fixed by using EnumerableLimit#create method, > instead of EnumerableLimit's constructor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2941) EnumerableLimitRule on Sort with no collation creates EnumerableLimit with wrong traitSet and cluster
[ https://issues.apache.org/jira/browse/CALCITE-2941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799154#comment-16799154 ] Ruben Quesada Lopez commented on CALCITE-2941: -- Sure, I'll add a test as soon as I can. > EnumerableLimitRule on Sort with no collation creates EnumerableLimit with > wrong traitSet and cluster > - > > Key: CALCITE-2941 > URL: https://issues.apache.org/jira/browse/CALCITE-2941 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.18.0 >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 10m > Remaining Estimate: 0h > > EnumerableLimitRule _"converst a Sort that has offset or fetch set to an > EnumerableLimit on top of a 'pure' Sort that has no offset or fetch"_. This > is the "normal" scenario, and there is no issue with that. > However, there is another scenario, where the EnumerableLimitRule is applied > on an "empty" Sort (with no field collations) with offset / fetch. In that > case, the EnumerableLimit's input will not be the Sort (because it will > disappear, since it was empty), but the Sort's input. The problem comes in > this case, because the EnumerableLimit will be created with the Sort's > traitSet and cluster; instead of the Sort's input (which is the actual > EnumerableLimit input) traitSet and cluster. > Probably the bug can be easily fixed by using EnumerableLimit#create method, > instead of EnumerableLimit's constructor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-883) Support RESPECT NULLS, IGNORE NULLS option for LEAD, LAG, FIRST_VALUE, LAST_VALUE window functions
[ https://issues.apache.org/jira/browse/CALCITE-883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799142#comment-16799142 ] Hongze Zhang commented on CALCITE-883: -- To me your code pasted above makes sense already. Otherwise maybe we can just make change to the SqlNode "SqlWindow" (but it is maybe a little tricky since the tokens should be at the lhs of the window expansion). And I can share some code later since I'm now on a vacation. Basically my point is to at least avoid changes to SqlCall/SqlBasicCall if possible, to keep them simple and easy to construct. Do you think it is really necessary to add the field the these classes? > Support RESPECT NULLS, IGNORE NULLS option for LEAD, LAG, FIRST_VALUE, > LAST_VALUE window functions > -- > > Key: CALCITE-883 > URL: https://issues.apache.org/jira/browse/CALCITE-883 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Sean Hsuan-Yi Chu >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > This issue was brought up in Drill community: > (https://issues.apache.org/jira/browse/DRILL-3597) > For convenience, I copied and pasted the reporter Khurram Faraaz's > description here: > "The SQL standard defines a RESPECT NULLS or IGNORE NULLS option for lead, > lag, first_value, and last_value window functions." > In Calcite, we should have been able to recognized this option. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2815) GeodeSchemaFactory change to pass in clientCache using geode jndi context
[ https://issues.apache.org/jira/browse/CALCITE-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799071#comment-16799071 ] Sandeep Chada commented on CALCITE-2815: In case we are okay having non-serializable objects in Schema constructor. Can we please review this PR https://github.com/apache/calcite/pull/1071? > GeodeSchemaFactory change to pass in clientCache using geode jndi context > - > > Key: CALCITE-2815 > URL: https://issues.apache.org/jira/browse/CALCITE-2815 > Project: Calcite > Issue Type: Improvement > Components: geode >Reporter: Sandeep Chada >Assignee: Andrei Sereda >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > We need ability to pass clientCache to GeodeSchemaFactory create static > function. In our application we already have clientCache pre created. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-883) Support RESPECT NULLS, IGNORE NULLS option for LEAD, LAG, FIRST_VALUE, LAST_VALUE window functions
[ https://issues.apache.org/jira/browse/CALCITE-883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799028#comment-16799028 ] Chunwei Lei edited comment on CALCITE-883 at 3/22/19 1:46 PM: -- I am sorry that you may still misunderstand me. I don't intend to prevent "nested" calls. I just wonder which way can work and also the best. Would you mind pasting some code to clarify your idea? was (Author: chunwei lei): I am sorry that you may still misunderstand me. I don't intent to prevent "nested" calls. I just wonder which way can work and also the best. Would you mind pasting some code to clarify your idea? > Support RESPECT NULLS, IGNORE NULLS option for LEAD, LAG, FIRST_VALUE, > LAST_VALUE window functions > -- > > Key: CALCITE-883 > URL: https://issues.apache.org/jira/browse/CALCITE-883 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Sean Hsuan-Yi Chu >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > This issue was brought up in Drill community: > (https://issues.apache.org/jira/browse/DRILL-3597) > For convenience, I copied and pasted the reporter Khurram Faraaz's > description here: > "The SQL standard defines a RESPECT NULLS or IGNORE NULLS option for lead, > lag, first_value, and last_value window functions." > In Calcite, we should have been able to recognized this option. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-883) Support RESPECT NULLS, IGNORE NULLS option for LEAD, LAG, FIRST_VALUE, LAST_VALUE window functions
[ https://issues.apache.org/jira/browse/CALCITE-883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799028#comment-16799028 ] Chunwei Lei commented on CALCITE-883: - I am sorry that you may still misunderstand me. I don't intent to prevent "nested" calls. I just wonder which way can work and also the best. Would you mind pasting some code to clarify your idea? > Support RESPECT NULLS, IGNORE NULLS option for LEAD, LAG, FIRST_VALUE, > LAST_VALUE window functions > -- > > Key: CALCITE-883 > URL: https://issues.apache.org/jira/browse/CALCITE-883 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Sean Hsuan-Yi Chu >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > This issue was brought up in Drill community: > (https://issues.apache.org/jira/browse/DRILL-3597) > For convenience, I copied and pasted the reporter Khurram Faraaz's > description here: > "The SQL standard defines a RESPECT NULLS or IGNORE NULLS option for lead, > lag, first_value, and last_value window functions." > In Calcite, we should have been able to recognized this option. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2945) use equals in Boolean object compare
[ https://issues.apache.org/jira/browse/CALCITE-2945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-2945: Labels: pull-request-available (was: ) > use equals in Boolean object compare > > > Key: CALCITE-2945 > URL: https://issues.apache.org/jira/browse/CALCITE-2945 > Project: Calcite > Issue Type: Improvement > Components: avatica >Affects Versions: 1.16.0, 1.17.0, 1.18.0 >Reporter: leesf >Assignee: leesf >Priority: Minor > Labels: pull-request-available > Fix For: next > > > In ConnectionPropertiesImpl#merge funciton, we could use equals method in > Boolean object compare. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2945) use equals in Boolean object compare
leesf created CALCITE-2945: -- Summary: use equals in Boolean object compare Key: CALCITE-2945 URL: https://issues.apache.org/jira/browse/CALCITE-2945 Project: Calcite Issue Type: Improvement Components: avatica Affects Versions: 1.18.0, 1.17.0, 1.16.0 Reporter: leesf Assignee: leesf Fix For: next In ConnectionPropertiesImpl#merge funciton, we could use equals method in Boolean object compare. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (CALCITE-2939) NPE Array Type serialToJdbc in TypedValue when executeBatch
[
https://issues.apache.org/jira/browse/CALCITE-2939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan reassigned CALCITE-2939:
---
Assignee: Danny Chan
> NPE Array Type serialToJdbc in TypedValue when executeBatch
> ---
>
> Key: CALCITE-2939
> URL: https://issues.apache.org/jira/browse/CALCITE-2939
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: 1.12.0
>Reporter: shining
>Assignee: Danny Chan
>Priority: Major
>
> I am using phoenix QueryServer upsert data in batch，the code as follows:
> {code:java}
> @Test
> public void preparedStatementArrayTest() throws Exception {
> final String tableName = "TEST";
> try (Connection conn =
> DriverManager.getConnection("jdbc:phoenix:thin:http://localhost:8765;);
> Statement stmt = conn.createStatement()) {
> conn.setAutoCommit(false);
> assertFalse(stmt.execute("DROP TABLE IF EXISTS " + tableName));
> assertFalse(stmt.execute("CREATE TABLE " + tableName + " ("
> + "pk VARCHAR NOT NULL PRIMARY KEY, "
> + "histogram INTEGER[])")
> );
> conn.commit();
> int numRows = 10;
> int numEvenElements = 4;
> int numOddElements = 6;
> try (PreparedStatement pstmt = conn.prepareStatement("UPSERT INTO " +
> tableName + " values(?, ?)")) {
> for (int i = 0; i < numRows; i++) {
> pstmt.setString(1, Integer.toString(i));
> int arrayLength = i % 2 == 0 ? numEvenElements : numOddElements;
> Object[] arrayData = new Object[arrayLength];
> for (int arrayOffset = 0; arrayOffset < arrayLength;
> arrayOffset++) {
> arrayData[arrayOffset] =
> Integer.toString(getArrayValueForOffset(arrayOffset));
> }
> pstmt.setArray(2, conn.createArrayOf("VARCHAR", arrayData));
> pstmt.addBatch();
> pstmt.executeBatch();
> }
> conn.commit();
> }
> }
> {code}
> When encounter Array type, I get an error NPE：
> {code:java}
> java.lang.NullPointerException
> at
> org.apache.calcite.avatica.remote.TypedValue.serialToJdbc(TypedValue.java:362)
> at
> org.apache.calcite.avatica.remote.TypedValue.protoToJdbc(TypedValue.java:895)
> at
> org.apache.calcite.avatica.jdbc.JdbcMeta.executeBatchProtobuf(JdbcMeta.java:986)
> at
> org.apache.calcite.avatica.remote.LocalService.apply(LocalService.java:363)
> at
> org.apache.calcite.avatica.remote.Service$ExecuteBatchRequest.accept(Service.java:2990)
> at
> org.apache.calcite.avatica.remote.Service$ExecuteBatchRequest.accept(Service.java:2942)
> at
> org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:94)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2741) Add operator table with Hive-specific built-in functions
[ https://issues.apache.org/jira/browse/CALCITE-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lai Zhou updated CALCITE-2741: -- Description: [~julianhyde], I write a hive adapter for calcite to support Hive sql ,includes UDF、UDAF、UDTF and some of SqlSpecialOperator. How do you think of supporting a direct implemention of hive sql like this? I think it will be valueable when someone want to migrate his hive etl jobs to real-time scene. was: [~julianhyde], I extended the native enummerable implemention of calcite to support Hive sql ,include UDF、UDAF and all the SqlSpecialOperator,which inspired by apache Drills. I modified the parser,type systems,and bridge the hive operator . How do you think of supporting a direct implemention of hive sql like this? I think it will be valueable when someone want to migrate his hive etl jobs to real-time scene. > Add operator table with Hive-specific built-in functions > > > Key: CALCITE-2741 > URL: https://issues.apache.org/jira/browse/CALCITE-2741 > Project: Calcite > Issue Type: New Feature > Components: core >Reporter: Lai Zhou >Priority: Minor > > [~julianhyde], > I write a hive adapter for calcite to support Hive sql ,includes > UDF、UDAF、UDTF and some of SqlSpecialOperator. > How do you think of supporting a direct implemention of hive sql like this? > I think it will be valueable when someone want to migrate his hive etl jobs > to real-time scene. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2741) Add operator table with Hive-specific built-in functions
[ https://issues.apache.org/jira/browse/CALCITE-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lai Zhou updated CALCITE-2741: -- Attachment: (was: 屏幕快照 2019-01-09 16.49.34.png) > Add operator table with Hive-specific built-in functions > > > Key: CALCITE-2741 > URL: https://issues.apache.org/jira/browse/CALCITE-2741 > Project: Calcite > Issue Type: New Feature > Components: core >Reporter: Lai Zhou >Priority: Minor > > [~julianhyde], > I write a hive adapter for calcite to support Hive sql ,includes > UDF、UDAF、UDTF and some of SqlSpecialOperator. > How do you think of supporting a direct implemention of hive sql like this? > I think it will be valueable when someone want to migrate his hive etl jobs > to real-time scene. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2940) CalcitePrepareImpl always creates a default SqlParser without user's CalciteConnectionConfig
[
https://issues.apache.org/jira/browse/CALCITE-2940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16798806#comment-16798806
]
Stamatis Zampetakis commented on CALCITE-2940:
--
Duplicate of which? Can you please add also the appropriate relationship?
> CalcitePrepareImpl always creates a default SqlParser without user's
> CalciteConnectionConfig
> --
>
> Key: CALCITE-2940
> URL: https://issues.apache.org/jira/browse/CALCITE-2940
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: Lai Zhou
>Priority: Major
>
> CalcitePrepareImpl 's method
> {code:java}
> private ParseResult parse_(Context context, String sql, boolean convert,
> boolean analyze, boolean fail)
> {code}
> should use the context to create a SqlParser.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2944) Lack of test for Aggregate.indicator
[ https://issues.apache.org/jira/browse/CALCITE-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16798803#comment-16798803 ] Stamatis Zampetakis commented on CALCITE-2944: -- Based on the existing comment I was thinking that indicator field should be rather deprecated. > Lack of test for Aggregate.indicator > > > Key: CALCITE-2944 > URL: https://issues.apache.org/jira/browse/CALCITE-2944 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Priority: Major > Attachments: image-2019-03-21-18-19-58-933.png > > > Although Aggregate.indicator is discouraged to use, but no test cases > guarantee indicator can work correctly. I tried to set indicator to always > false in Aggregate constructor, it passed all the tests. > !image-2019-03-21-18-19-58-933.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2943) Materialized view rewriting logic calls getApplicableMaterializations each time the rule is triggered
[
https://issues.apache.org/jira/browse/CALCITE-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16798796#comment-16798796
]
Stamatis Zampetakis commented on CALCITE-2943:
--
Nice screenshot! Which program did you use to get it?
> Materialized view rewriting logic calls getApplicableMaterializations each
> time the rule is triggered
> -
>
> Key: CALCITE-2943
> URL: https://issues.apache.org/jira/browse/CALCITE-2943
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: Screen Shot 2019-03-21 at 2.33.01 PM.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{RelOptMaterializations.getApplicableMaterializations}} is called each time
> the rule is triggered.
> {code:java}
> ...
> // Obtain applicable (filtered) materializations
> // TODO: Filtering of relevant materializations needs to be
> // improved so we gather only materializations that might
> // actually generate a valid rewriting.
> final List applicableMaterializations =
> RelOptMaterializations.getApplicableMaterializations(node,
> materializations);
> ...
> {code}
> When I implemented the rule, I assumed (incorrectly) that
> {{getApplicableMaterializations}} was a lightweight call and hence would help
> discarding materialized views extracted from the planner quickly. It turns
> out that the method can quickly become the most time consuming part of the
> rule execution; I assume the method was just supposed to be used once per
> query.
> !Screen Shot 2019-03-21 at 2.33.01 PM.png!
> Since the prefiltering that we do right now is rather simple and we already
> extract the tables used by queries and materializations within the rule, we
> can just skip the materialization over there.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
