[jira] [Created] (CALCITE-2222) Add Quarter as part of valid floor units to push down to Druid.
slim bouguerra created CALCITE-:
---
Summary: Add Quarter as part of valid floor units to push down to
Druid.
Key: CALCITE-
URL: https://issues.apache.org/jira/browse/CALCITE-
Project: Calcite
Issue Type: Bug
Components: druid
Affects Versions: 1.16.0
Reporter: slim bouguerra
Assignee: slim bouguerra
Fix For: 1.17.0
Current list of valid floor units that can be pushed as valid floor unit is
missing Quarters, something that Druid can actually do.
This is a performance bug.
For instance query
{code}
SELECT floor_year(`__time`), max(added), sum(variation)
FROM druid_table_1
{code}
Is currently planned as
{code}
{"queryType":"scan","dataSource":"wikipedia","intervals":["1900-01-01T00:00:00.000Z/3000-01-01T00:00:00.000Z"],"virtualColumns":[{"type":"expression","name":"vc","expression":"\"__time\"","outputType":"LONG"}],"columns":["vc","added","variation"],"resultFormat":"compactedList"}
{code}
And it can be optimized to
{code}
{"queryType":"timeseries","dataSource":"wikipedia","descending":false,"granularity":"quarter","aggregations":[{"type":"doubleMax","name":"$f1","fieldName":"added"},{"type":"doubleSum","name":"$f2","fieldName":"variation"}],"intervals":["1900-01-01T00:00:00.000Z/3000-01-01T00:00:00.000Z"],"context":{"skipEmptyBuckets":true}}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2205) One more Infinite loop for JoinPushTransitivePredicatesRule
[
https://issues.apache.org/jira/browse/CALCITE-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409877#comment-16409877
]
Vitalii Diravka commented on CALCITE-2205:
--
[~julianhyde] I rethought my approach and looked into _RelOptPredicateList_, as
you suggested.
There were redundant inferred predicates: some of them appeared, because on the
one side of join were simplified predicates (after filter creating) and on
other side there were not simplified ones. And they were detected as new
predicates.
Also not all predicates were filtered in the _RelMdPredicates#infer_ .
Also I have noticed that _RelMetadataTest$testPullUpPredicatesForExprsItr_
passes faster and there are much less duplicates in the _pulledUpPredicates_
My branch is rebased onto last Calcite master branch. Please review the updated
[PR-645|https://github.com/apache/calcite/pull/645] and let me know if you have
some questions.
> One more Infinite loop for JoinPushTransitivePredicatesRule
> ---
>
> Key: CALCITE-2205
> URL: https://issues.apache.org/jira/browse/CALCITE-2205
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.15.0
>Reporter: Vitalii Diravka
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.17.0
>
>
> CALCITE-2200 resolves some cases of infinite loop via stopping of recursion
> in HepPlanner#applyRules, when newVertex is the same as vertex.
> In this jira one more case of infinite loop is described:
> JoinPushTransitivePredicatesRule#onMatch generates new right or left inputs
> via using RelBuilder#filter method on top of LogicalFilter RelNode with the
> same condition.
> In this case a new RelNode shouldn't be created. Possible fix to change logic
> of RelBuilder#filter method.
> TestCase for reproduce:
> {code}
> @Test public void testJoinPushTransitivePredicatesRule2() {
> HepProgramBuilder builder = new HepProgramBuilder();
> builder.addRuleInstance(JoinPushTransitivePredicatesRule.INSTANCE);
> HepProgram build = builder.build();
> HepPlanner hepPlanner = new HepPlanner(build);
> final String sql = "select n1.SAL from EMPNULLABLES_20 n1 where n1.SAL\n"
> + "IN (select n2.SAL from EMPNULLABLES_20 n2 "
> + "where n1.SAL = n2.SAL or n1.SAL = 4)";
> sql(sql)
> .withDecorrelation(true)
> .with(hepPlanner)
> .check();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2221) RelRoot.project does not take into account flattened record type
[
https://issues.apache.org/jira/browse/CALCITE-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shuyi Chen updated CALCITE-2221:
Description:
To reproduce, simply run:
@Test public void testFlattenRecords() {
sql("select employees[1] from dept_nested").ok();
}
It will yield:
LogicalProject(EXPR$0=[$0])
LogicalProject(EXPR$0$0=[ITEM($2, 1).EMPNO], EXPR$0$1=[ITEM($2, 1).ENAME],
EXPR$0$2=[ITEM($2, 1).DETAIL])
LogicalTableScan(table=[[CATALOG, SALES, DEPT_NESTED]])
was:
To reproduce, simply run:
@Test public void testFlattenRecords() {
sql("select employees[1] from dept_nested").ok();
}
It will yield:
LogicalProject(EXPR$0=[$0])
LogicalProject(EXPR$0$0=[ITEM($6, 1).EMPNO], EXPR$0$1=[ITEM($6, 1).ENAME],
EXPR$0$2=[ITEM($6, 1).DETAIL])
LogicalProject(DEPTNO=[$0], NAME=[$1], TYPE=[$2.TYPE], DESC=[$2.DESC],
A=[$2.OTHERS.A], B=[$2.OTHERS.B], EMPLOYEES=[$3])
LogicalTableScan(table=[[CATALOG, SALES, DEPT_NESTED]])
> RelRoot.project does not take into account flattened record type
>
>
> Key: CALCITE-2221
> URL: https://issues.apache.org/jira/browse/CALCITE-2221
> Project: Calcite
> Issue Type: Task
> Components: core
>Reporter: Shuyi Chen
>Assignee: Julian Hyde
>Priority: Major
>
> To reproduce, simply run:
> @Test public void testFlattenRecords() {
> sql("select employees[1] from dept_nested").ok();
> }
> It will yield:
> LogicalProject(EXPR$0=[$0])
> LogicalProject(EXPR$0$0=[ITEM($2, 1).EMPNO], EXPR$0$1=[ITEM($2, 1).ENAME],
> EXPR$0$2=[ITEM($2, 1).DETAIL])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT_NESTED]])
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2221) RelRoot.project does not take into account flattened record type
Shuyi Chen created CALCITE-2221:
---
Summary: RelRoot.project does not take into account flattened
record type
Key: CALCITE-2221
URL: https://issues.apache.org/jira/browse/CALCITE-2221
Project: Calcite
Issue Type: Task
Components: core
Reporter: Shuyi Chen
Assignee: Julian Hyde
To reproduce, simply run:
@Test public void testFlattenRecords() {
sql("select employees[1] from dept_nested").ok();
}
It will yield:
LogicalProject(EXPR$0=[$0])
LogicalProject(EXPR$0$0=[ITEM($6, 1).EMPNO], EXPR$0$1=[ITEM($6, 1).ENAME],
EXPR$0$2=[ITEM($6, 1).DETAIL])
LogicalProject(DEPTNO=[$0], NAME=[$1], TYPE=[$2.TYPE], DESC=[$2.DESC],
A=[$2.OTHERS.A], B=[$2.OTHERS.B], EMPLOYEES=[$3])
LogicalTableScan(table=[[CATALOG, SALES, DEPT_NESTED]])
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2220) Wrong ordinal when flattening multiple input records during converting to REL
[
https://issues.apache.org/jira/browse/CALCITE-2220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409204#comment-16409204
]
Shuyi Chen commented on CALCITE-2220:
-
[~julianhyde] , can you help take a look at this JIRA and PR?
> Wrong ordinal when flattening multiple input records during converting to REL
> -
>
> Key: CALCITE-2220
> URL: https://issues.apache.org/jira/browse/CALCITE-2220
> Project: Calcite
> Issue Type: Task
> Components: core
>Reporter: Shuyi Chen
>Assignee: Shuyi Chen
>Priority: Major
>
> *Issue can be reproduced as follows*:
> MockTable deptNestedTable =
> MockTable.create(this, salesSchema, "DEPT_NESTED", false, 4);
> deptNestedTable.addColumn("DEPTNO", f.intType, true);
> deptNestedTable.addColumn("NAME", f.varchar10Type);
> deptNestedTable.addColumn("SKILLRECORD", f.skillRecordType);
> deptNestedTable.addColumn("EMPLOYEES", f.empListType);
> registerTable(deptNestedTable);
> Run the following test:
> @Test public void testArrayOfRecord() {
> sql("select employees[1] from dept_nested").ok();
> }
> The following error gets thrown when run:
> java.lang.AssertionError: type mismatch:
> ref:
> RecordType(INTEGER NOT NULL EMPNO, VARCHAR(10) CHARACTER SET "ISO-8859-1"
> COLLATE "ISO-8859-1$en_US$primary" NOT NULL ENAME, RecordType(VARCHAR(10)
> CHARACTER SET "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary" NOT NULL TYPE,
> VARCHAR(20) CHARACTER SET "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary" NOT
> NULL DESC) NOT NULL ARRAY NOT NULL SKILLS) NOT NULL ARRAY NOT NULL
> input:
> VARCHAR(20) CHARACTER SET "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary" NOT
> NULL
> at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31)
> at org.apache.calcite.plan.RelOptUtil.eq(RelOptUtil.java:1838)
> at org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:125)
> at org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:57)
> at org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:112)
> at org.apache.calcite.rex.RexChecker.visitCall(RexChecker.java:140)
> at org.apache.calcite.rex.RexChecker.visitCall(RexChecker.java:57)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:107)
> at
> org.apache.calcite.rex.RexVisitorImpl.visitFieldAccess(RexVisitorImpl.java:98)
> at
> org.apache.calcite.rex.RexChecker.visitFieldAccess(RexChecker.java:149)
> at
> org.apache.calcite.rex.RexChecker.visitFieldAccess(RexChecker.java:57)
> at org.apache.calcite.rex.RexFieldAccess.accept(RexFieldAccess.java:81)
> at org.apache.calcite.rel.core.Project.isValid(Project.java:187)
> at org.apache.calcite.rel.core.Project.(Project.java:84)
> at
> org.apache.calcite.rel.logical.LogicalProject.(LogicalProject.java:65)
> at
> org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:120)
> at
> org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:103)
> at
> org.apache.calcite.rel.core.RelFactories$ProjectFactoryImpl.createProject(RelFactories.java:127)
> at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1064)
> at
> org.apache.calcite.plan.RelOptUtil.createProject(RelOptUtil.java:2956)
> at
> org.apache.calcite.plan.RelOptUtil.createProject(RelOptUtil.java:2873)
> at
> org.apache.calcite.sql2rel.RelStructuredTypeFlattener.rewriteRel(RelStructuredTypeFlattener.java:477)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.calcite.util.ReflectUtil.invokeVisitorInternal(ReflectUtil.java:257)
> at
> org.apache.calcite.util.ReflectUtil.invokeVisitor(ReflectUtil.java:214)
> at
> org.apache.calcite.util.ReflectUtil$1.invokeVisitor(ReflectUtil.java:464)
> at
> org.apache.calcite.sql2rel.RelStructuredTypeFlattener$RewriteRelVisitor.visit(RelStructuredTypeFlattener.java:721)
> at
> org.apache.calcite.sql2rel.RelStructuredTypeFlattener.rewrite(RelStructuredTypeFlattener.java:177)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.flattenTypes(SqlToRelConverter.java:462)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:585)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.assertConvertsTo(SqlToRelTestBase.java:690)
> at
> org.apache.calcite.test.SqlToRelConverterTest$Sql.convertsTo(SqlToRelConverterTest.java:2784)
> at
>
[jira] [Created] (CALCITE-2220) Wrong ordinal when flattening multiple input records during converting to REL
Shuyi Chen created CALCITE-2220:
---
Summary: Wrong ordinal when flattening multiple input records
during converting to REL
Key: CALCITE-2220
URL: https://issues.apache.org/jira/browse/CALCITE-2220
Project: Calcite
Issue Type: Task
Components: core
Reporter: Shuyi Chen
Assignee: Shuyi Chen
*Issue can be reproduced as follows*:
MockTable deptNestedTable =
MockTable.create(this, salesSchema, "DEPT_NESTED", false, 4);
deptNestedTable.addColumn("DEPTNO", f.intType, true);
deptNestedTable.addColumn("NAME", f.varchar10Type);
deptNestedTable.addColumn("SKILLRECORD", f.skillRecordType);
deptNestedTable.addColumn("EMPLOYEES", f.empListType);
registerTable(deptNestedTable);
Run the following test:
@Test public void testArrayOfRecord() {
sql("select employees[1] from dept_nested").ok();
}
The following error gets thrown when run:
java.lang.AssertionError: type mismatch:
ref:
RecordType(INTEGER NOT NULL EMPNO, VARCHAR(10) CHARACTER SET "ISO-8859-1"
COLLATE "ISO-8859-1$en_US$primary" NOT NULL ENAME, RecordType(VARCHAR(10)
CHARACTER SET "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary" NOT NULL TYPE,
VARCHAR(20) CHARACTER SET "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary" NOT
NULL DESC) NOT NULL ARRAY NOT NULL SKILLS) NOT NULL ARRAY NOT NULL
input:
VARCHAR(20) CHARACTER SET "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary" NOT
NULL
at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31)
at org.apache.calcite.plan.RelOptUtil.eq(RelOptUtil.java:1838)
at org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:125)
at org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:57)
at org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:112)
at org.apache.calcite.rex.RexChecker.visitCall(RexChecker.java:140)
at org.apache.calcite.rex.RexChecker.visitCall(RexChecker.java:57)
at org.apache.calcite.rex.RexCall.accept(RexCall.java:107)
at
org.apache.calcite.rex.RexVisitorImpl.visitFieldAccess(RexVisitorImpl.java:98)
at
org.apache.calcite.rex.RexChecker.visitFieldAccess(RexChecker.java:149)
at
org.apache.calcite.rex.RexChecker.visitFieldAccess(RexChecker.java:57)
at org.apache.calcite.rex.RexFieldAccess.accept(RexFieldAccess.java:81)
at org.apache.calcite.rel.core.Project.isValid(Project.java:187)
at org.apache.calcite.rel.core.Project.(Project.java:84)
at
org.apache.calcite.rel.logical.LogicalProject.(LogicalProject.java:65)
at
org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:120)
at
org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:103)
at
org.apache.calcite.rel.core.RelFactories$ProjectFactoryImpl.createProject(RelFactories.java:127)
at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1064)
at
org.apache.calcite.plan.RelOptUtil.createProject(RelOptUtil.java:2956)
at
org.apache.calcite.plan.RelOptUtil.createProject(RelOptUtil.java:2873)
at
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.rewriteRel(RelStructuredTypeFlattener.java:477)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.calcite.util.ReflectUtil.invokeVisitorInternal(ReflectUtil.java:257)
at
org.apache.calcite.util.ReflectUtil.invokeVisitor(ReflectUtil.java:214)
at
org.apache.calcite.util.ReflectUtil$1.invokeVisitor(ReflectUtil.java:464)
at
org.apache.calcite.sql2rel.RelStructuredTypeFlattener$RewriteRelVisitor.visit(RelStructuredTypeFlattener.java:721)
at
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.rewrite(RelStructuredTypeFlattener.java:177)
at
org.apache.calcite.sql2rel.SqlToRelConverter.flattenTypes(SqlToRelConverter.java:462)
at
org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:585)
at
org.apache.calcite.test.SqlToRelTestBase$TesterImpl.assertConvertsTo(SqlToRelTestBase.java:690)
at
org.apache.calcite.test.SqlToRelConverterTest$Sql.convertsTo(SqlToRelConverterTest.java:2784)
at
org.apache.calcite.test.SqlToRelConverterTest$Sql.ok(SqlToRelConverterTest.java:2776)
at
org.apache.calcite.test.SqlToRelConverterTest.testArrayOfRecord(SqlToRelConverterTest.java:1059)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
