[jira] [Commented] (CALCITE-2116) The digests are not same for the common sub-expressions in HepPlanner
[
https://issues.apache.org/jira/browse/CALCITE-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317914#comment-16317914
]
Julian Hyde commented on CALCITE-2116:
--
Oh, I see now. You had to change Interpreter because now the plan is a DAG, a
Node can have more than one consumer.
A better fix would be to insert a new kind of Sink, say DuplicatingSink, that
copies each incoming row to each consumer. If a node has only one consumer it
would continue to use ListSink. Let me try and write that.
> The digests are not same for the common sub-expressions in HepPlanner
> -
>
> Key: CALCITE-2116
> URL: https://issues.apache.org/jira/browse/CALCITE-2116
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>
> The digests are not same for the same sub-query.
> like query
> {code:java}
> select * from (select * from dept union all select * from dept)a;
> {code}
> When run the query, The union relNode has two inputs, and it's same expectly
> and it's the same object instead of the print of plan, but Now they are not
> same.
> And for Test testReplaceCommonSubexpression, the rule is only firing once,
> but now it's two.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2123) Bug in the Druid Filter Translation when Comparing String Ref to a Constant Number
[
https://issues.apache.org/jira/browse/CALCITE-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317689#comment-16317689
]
slim bouguerra commented on CALCITE-2123:
-
okay Thanks, will explore that.
> Bug in the Druid Filter Translation when Comparing String Ref to a Constant
> Number
> --
>
> Key: CALCITE-2123
> URL: https://issues.apache.org/jira/browse/CALCITE-2123
> Project: Calcite
> Issue Type: Bug
> Components: druid
>Reporter: slim bouguerra
>Assignee: Julian Hyde
>
> The following query {code} SELECT COUNT(*) FROM \"foodmart\" WHERE
> \"product_id\" = 16.0{code} Translates to a Druid Table Scan with a String to
> String Selector comparison filter.
> instead we need to have a Bound filter that cast the String to number.
> This is what we should expect.
> {code}
> {"type":"bound","dimension":"product_id","lower":"16.0","lowerStrict":false,"upper":"16.0","upperStrict":false,"ordering":"numeric"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2116) The digests are not same for the common sub-expressions in HepPlanner
[
https://issues.apache.org/jira/browse/CALCITE-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317670#comment-16317670
]
Julian Hyde commented on CALCITE-2116:
--
Reviewing
[721f387|https://github.com/apache/calcite/pull/597/commits/721f3879a9c9bcf7033afab927f30e6b9a2d69d4].
Did you intend to include Interpreter.java in the PR? Other than that, looks
good.
> The digests are not same for the common sub-expressions in HepPlanner
> -
>
> Key: CALCITE-2116
> URL: https://issues.apache.org/jira/browse/CALCITE-2116
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>
> The digests are not same for the same sub-query.
> like query
> {code:java}
> select * from (select * from dept union all select * from dept)a;
> {code}
> When run the query, The union relNode has two inputs, and it's same expectly
> and it's the same object instead of the print of plan, but Now they are not
> same.
> And for Test testReplaceCommonSubexpression, the rule is only firing once,
> but now it's two.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2125) error plan result that using AggregateExpandDistinctAggregatesRule.INSTANCE and AggregateProjectMergeRule.INSTANCE
[
https://issues.apache.org/jira/browse/CALCITE-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317653#comment-16317653
]
Julian Hyde commented on CALCITE-2125:
--
I see. Thanks for clarifying.
> error plan result that using AggregateExpandDistinctAggregatesRule.INSTANCE
> and AggregateProjectMergeRule.INSTANCE
> ---
>
> Key: CALCITE-2125
> URL: https://issues.apache.org/jira/browse/CALCITE-2125
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.15.0
>Reporter: godfrey he
>Assignee: Julian Hyde
>
> test case:
> {code}
> @Test public void testDistinctCount0() {
> final HepProgram program = HepProgram.builder()
> .addRuleInstance(AggregateExpandDistinctAggregatesRule.INSTANCE)
> .addRuleInstance(AggregateProjectMergeRule.INSTANCE)
> .build();
> checkPlanning(program,
> "select type, count(distinct acctno), sum(distinct balance)"
> + " from customer.account group by type");
> }
> {code}
> current result:
> {code}
>
>
>
>
>
>
>
>
>
>
>
> {code}
> However, the result plan is wrong.
> first, if we only use *AggregateExpandDistinctAggregatesRule.INSTANCE* to
> optimize the query, the result plan is correct:
> {code}
> @Test public void testDistinctCount0() {
> final HepProgram program = HepProgram.builder()
> .addRuleInstance(AggregateExpandDistinctAggregatesRule.INSTANCE)
> //.addRuleInstance(AggregateProjectMergeRule.INSTANCE)
> .build();
> checkPlanning(program,
> "select type, count(distinct acctno), sum(distinct balance)"
> + " from customer.account group by type");
> }
> LogicalProject(TYPE=[$0], EXPR$1=[$1], EXPR$2=[CAST($2):INTEGER NOT NULL])
> LogicalAggregate(group=[{0}], EXPR$1=[COUNT($1) FILTER $3], EXPR$2=[SUM($2)
> FILTER $4])
> LogicalProject(TYPE=[$0], ACCTNO=[$1], BALANCE=[$2], $g_1=[=($3, 1)],
> $g_2=[=($3, 2)])
> LogicalAggregate(group=[{0, 1, 2}], groups=[[{0, 1}, {0, 2}]],
> $g=[GROUPING($0, $1, $2)])
> LogicalProject(TYPE=[$1], ACCTNO=[$0], BALANCE=[$2])
> LogicalTableScan(table=[[CATALOG, CUSTOMER, ACCOUNT]])
> {code}
> then *AggregateProjectMergeRule.INSTANCE* is added, it will change the
> sub-plan from
> {code}
> LogicalAggregate(group=[{0, 1, 2}], groups=[[{0, 1}, {0, 2}]],
> $g=[GROUPING($0, $1, $2)])
> LogicalProject(TYPE=[$1], ACCTNO=[$0], BALANCE=[$2])
> {code}
> to
> {code}
> LogicalAggregate(group=[{0, 1, 2}], groups=[[{0, 1}, {1, 2}]],
> $g=[GROUPING($1, $0, $2)])
> {code}
> Notes that the groups was changed from {code} groups=[[{0, 1}, {0, 2}]]
> {code} to
> {code} groups=[[{0, 1}, {1, 2}]] {code}, but the filter values generated by
> *groupValue* in *AggregateExpandDistinctAggregatesRule* are not changed in
> project:
> {code}
> LogicalProject(TYPE=[$0], ACCTNO=[$1], BALANCE=[$2], $g_1=[=($3, 1)],
> $g_2=[=($3, 2)])
> {code}
> {code}
> // filter values before AggregateProjectMergeRule added
> AggregateExpandDistinctAggregatesRule.groupValue(ImmutableBitSet.of(0,1,2),
> ImmutableBitSet.of(0,1)) is 1
> AggregateExpandDistinctAggregatesRule.groupValue(ImmutableBitSet.of(0,1,2),
> ImmutableBitSet.of(0,2)) is 2
> // filter values after AggregateProjectMergeRule added
> AggregateExpandDistinctAggregatesRule.groupValue(ImmutableBitSet.of(1,0,2),
> ImmutableBitSet.of(0,1)) is 1
> AggregateExpandDistinctAggregatesRule.groupValue(ImmutableBitSet.of(1,0,2),
> ImmutableBitSet.of(1,2)) is 4
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2032) Error when implicitly converting character literal to date literal
[
https://issues.apache.org/jira/browse/CALCITE-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317549#comment-16317549
]
Julian Hyde commented on CALCITE-2032:
--
CALCITE-613 dealt with {{numeric = string}}, whereas this case deals with
{{date = string}} but I suspect that the solution is similar.
> Error when implicitly converting character literal to date literal
> --
>
> Key: CALCITE-2032
> URL: https://issues.apache.org/jira/browse/CALCITE-2032
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Trivial
>
> The following query implicitly converts a character literal to a date
> literal; it passes validation but fails during code generation.
> {code}
> select *
> from "scott".emp
> WHERE hiredate = '1980-12-17';
> +---+---+---+--+++--++
> | EMPNO | ENAME | JOB | MGR | HIREDATE | SAL| COMM | DEPTNO |
> +---+---+---+--+++--++
> | 7369 | SMITH | CLERK | 7902 | 1980-12-17 | 800.00 | | 20 |
> +---+---+---+--+++--++
> (1 row)
> !ok
> {code}
> throws
> {code}
> Error while compiling generated Java code:
> at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
> at
> org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:108)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1261)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:330)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:229)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:786)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:221)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
> ... 9 more
> Caused by: org.codehaus.commons.compiler.CompileException: Line 17, Column
> 46: Incomparable types 'java.lang.Integer' and 'java.lang.String'
> at
> org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:10092)
> at
> org.codehaus.janino.UnitCompiler.compileBoolean2(UnitCompiler.java:3080)
> {code}
> I have added the test case to {{misc.iq}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2032) Error when implicitly converting character literal to date literal
[
https://issues.apache.org/jira/browse/CALCITE-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317545#comment-16317545
]
Julian Hyde commented on CALCITE-2032:
--
I believe you're referring to [COMPARABLE_UNORDERED_COMPARABLE_UNORDERED
|https://github.com/apache/calcite/blob/a3dd90f1088966dc09caecf88779853ba34d290c/core/src/main/java/org/apache/calcite/sql/type/OperandTypes.java#L387].
This is for argument types that can be compared (i.e. you can say whether x =
y) but unordered (you can't say whether x > y or x < y). Right now we don't
have any such types in Calcite (search for uses of
{{RelDataTypeComparability.UNORDERED}} and you'll see) so it's moot.
> Error when implicitly converting character literal to date literal
> --
>
> Key: CALCITE-2032
> URL: https://issues.apache.org/jira/browse/CALCITE-2032
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Trivial
>
> The following query implicitly converts a character literal to a date
> literal; it passes validation but fails during code generation.
> {code}
> select *
> from "scott".emp
> WHERE hiredate = '1980-12-17';
> +---+---+---+--+++--++
> | EMPNO | ENAME | JOB | MGR | HIREDATE | SAL| COMM | DEPTNO |
> +---+---+---+--+++--++
> | 7369 | SMITH | CLERK | 7902 | 1980-12-17 | 800.00 | | 20 |
> +---+---+---+--+++--++
> (1 row)
> !ok
> {code}
> throws
> {code}
> Error while compiling generated Java code:
> at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
> at
> org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:108)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1261)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:330)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:229)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:786)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:221)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
> ... 9 more
> Caused by: org.codehaus.commons.compiler.CompileException: Line 17, Column
> 46: Incomparable types 'java.lang.Integer' and 'java.lang.String'
> at
> org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:10092)
> at
> org.codehaus.janino.UnitCompiler.compileBoolean2(UnitCompiler.java:3080)
> {code}
> I have added the test case to {{misc.iq}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2125) error plan result that using AggregateExpandDistinctAggregatesRule.INSTANCE and AggregateProjectMergeRule.INSTANCE
[
https://issues.apache.org/jira/browse/CALCITE-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317544#comment-16317544
]
godfrey he commented on CALCITE-2125:
-
actually, I should use grouping aggregate function to get filter values
instread of AggregateExpandDistinctAggregatesRule's groupValue.
{code}
private static long grouping(List groupingArgList, ImmutableBitSet
groupSet) {
int r = 0;
for(int f : groupingArgList) {
r = (r << 1) + (groupSet.toList().contains(f) ? 0 : 1);
}
return r;
}
// filter values before AggregateProjectMergeRule added
// LogicalAggregate(group=[{0, 1, 2}], groups=[[{0, 1}, {0, 2}]],
$g=[GROUPING($0, $1, $2)])
groupValue(ImmutableBitSet.of(0, 1, 2), ImmutableBitSet.of(0, 1)) is 1
grouping(Lists.newArrayList(0, 1, 2), ImmutableBitSet.of(0, 1)) is 1
groupValue(ImmutableBitSet.of(0, 1, 2), ImmutableBitSet.of(0, 2)) is 2
grouping(Lists.newArrayList(0, 1, 2), ImmutableBitSet.of(0, 2)) is 2
// filter values after AggregateProjectMergeRule added
// LogicalAggregate(group=[{0, 1, 2}], groups=[[{0, 1}, {1, 2}]],
$g=[GROUPING($1, $0, $2)])
groupValue(ImmutableBitSet.of(1,0,2), ImmutableBitSet.of(0,1)) is 1
grouping(Lists.newArrayList(1,0,2), ImmutableBitSet.of(0, 1)) is 1
groupValue(ImmutableBitSet.of(1, 0, 2), ImmutableBitSet.of(1, 2)) is 4
grouping(Lists.newArrayList(1,0,2), ImmutableBitSet.of(1, 2)) is 2
{code}
> error plan result that using AggregateExpandDistinctAggregatesRule.INSTANCE
> and AggregateProjectMergeRule.INSTANCE
> ---
>
> Key: CALCITE-2125
> URL: https://issues.apache.org/jira/browse/CALCITE-2125
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.15.0
>Reporter: godfrey he
>Assignee: Julian Hyde
>
> test case:
> {code}
> @Test public void testDistinctCount0() {
> final HepProgram program = HepProgram.builder()
> .addRuleInstance(AggregateExpandDistinctAggregatesRule.INSTANCE)
> .addRuleInstance(AggregateProjectMergeRule.INSTANCE)
> .build();
> checkPlanning(program,
> "select type, count(distinct acctno), sum(distinct balance)"
> + " from customer.account group by type");
> }
> {code}
> current result:
> {code}
>
>
>
>
>
>
>
>
>
>
>
> {code}
> However, the result plan is wrong.
> first, if we only use *AggregateExpandDistinctAggregatesRule.INSTANCE* to
> optimize the query, the result plan is correct:
> {code}
> @Test public void testDistinctCount0() {
> final HepProgram program = HepProgram.builder()
> .addRuleInstance(AggregateExpandDistinctAggregatesRule.INSTANCE)
> //.addRuleInstance(AggregateProjectMergeRule.INSTANCE)
> .build();
> checkPlanning(program,
> "select type, count(distinct acctno), sum(distinct balance)"
> + " from customer.account group by type");
> }
> LogicalProject(TYPE=[$0], EXPR$1=[$1], EXPR$2=[CAST($2):INTEGER NOT NULL])
> LogicalAggregate(group=[{0}], EXPR$1=[COUNT($1) FILTER $3], EXPR$2=[SUM($2)
> FILTER $4])
> LogicalProject(TYPE=[$0], ACCTNO=[$1], BALANCE=[$2], $g_1=[=($3, 1)],
> $g_2=[=($3, 2)])
> LogicalAggregate(group=[{0, 1, 2}], groups=[[{0, 1}, {0, 2}]],
> $g=[GROUPING($0, $1, $2)])
> LogicalProject(TYPE=[$1], ACCTNO=[$0], BALANCE=[$2])
> LogicalTableScan(table=[[CATALOG, CUSTOMER, ACCOUNT]])
> {code}
> then *AggregateProjectMergeRule.INSTANCE* is added, it will change the
> sub-plan from
> {code}
> LogicalAggregate(group=[{0, 1, 2}], groups=[[{0, 1}, {0, 2}]],
> $g=[GROUPING($0, $1, $2)])
> LogicalProject(TYPE=[$1], ACCTNO=[$0], BALANCE=[$2])
> {code}
> to
> {code}
> LogicalAggregate(group=[{0, 1, 2}], groups=[[{0, 1}, {1, 2}]],
> $g=[GROUPING($1, $0, $2)])
> {code}
> Notes that the groups was changed from {code} groups=[[{0, 1}, {0, 2}]]
> {code} to
> {code} groups=[[{0, 1}, {1, 2}]] {code}, but the filter values generated by
> *groupValue* in *AggregateExpandDistinctAggregatesRule* are not changed in
> project:
> {code}
> LogicalProject(TYPE=[$0], ACCTNO=[$1], BALANCE=[$2], $g_1=[=($3, 1)],
> $g_2=[=($3, 2)])
> {code}
> {code}
> // filter values before AggregateProjectMergeRule added
> AggregateExpandDistinctAggregatesRule.groupValue(ImmutableBitSet.of(0,1,2),
> ImmutableBitSet.of(0,1)) is 1
> AggregateExpandDistinctAggregatesRule.groupValue(ImmutableBitSet.of(0,1,2),
> ImmutableBitSet.of(0,2)) is 2
> // filter values after AggregateProjectMergeRule added
> AggregateExpandDistinctAggregatesRule.groupValue(ImmutableBitSet.of(1,0,2),
>
[jira] [Commented] (CALCITE-2123) Bug in the Druid Filter Translation when Comparing String Ref to a Constant Number
[
https://issues.apache.org/jira/browse/CALCITE-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317535#comment-16317535
]
Julian Hyde commented on CALCITE-2123:
--
I confess I'm confused by {{coerce}} too. However, from reading the comments,
it seems that {{coerce = true}} is the best thing.
Did you come across {{enum SqlOperandTypeChecker.Consistency}}, introduced in
CALCITE-613? In particular how {{StandardConvertletTable.consistentType}} uses
{{Consistency.COMPARE}}? I think this a better way to go than {{canCastFrom}}.
> Bug in the Druid Filter Translation when Comparing String Ref to a Constant
> Number
> --
>
> Key: CALCITE-2123
> URL: https://issues.apache.org/jira/browse/CALCITE-2123
> Project: Calcite
> Issue Type: Bug
> Components: druid
>Reporter: slim bouguerra
>Assignee: Julian Hyde
>
> The following query {code} SELECT COUNT(*) FROM \"foodmart\" WHERE
> \"product_id\" = 16.0{code} Translates to a Druid Table Scan with a String to
> String Selector comparison filter.
> instead we need to have a Bound filter that cast the String to number.
> This is what we should expect.
> {code}
> {"type":"bound","dimension":"product_id","lower":"16.0","lowerStrict":false,"upper":"16.0","upperStrict":false,"ordering":"numeric"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Closed] (CALCITE-2125) error plan result that using AggregateExpandDistinctAggregatesRule.INSTANCE and AggregateProjectMergeRule.INSTANCE
[
https://issues.apache.org/jira/browse/CALCITE-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
godfrey he closed CALCITE-2125.
---
Resolution: Not A Bug
> error plan result that using AggregateExpandDistinctAggregatesRule.INSTANCE
> and AggregateProjectMergeRule.INSTANCE
> ---
>
> Key: CALCITE-2125
> URL: https://issues.apache.org/jira/browse/CALCITE-2125
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.15.0
>Reporter: godfrey he
>Assignee: Julian Hyde
>
> test case:
> {code}
> @Test public void testDistinctCount0() {
> final HepProgram program = HepProgram.builder()
> .addRuleInstance(AggregateExpandDistinctAggregatesRule.INSTANCE)
> .addRuleInstance(AggregateProjectMergeRule.INSTANCE)
> .build();
> checkPlanning(program,
> "select type, count(distinct acctno), sum(distinct balance)"
> + " from customer.account group by type");
> }
> {code}
> current result:
> {code}
>
>
>
>
>
>
>
>
>
>
>
> {code}
> However, the result plan is wrong.
> first, if we only use *AggregateExpandDistinctAggregatesRule.INSTANCE* to
> optimize the query, the result plan is correct:
> {code}
> @Test public void testDistinctCount0() {
> final HepProgram program = HepProgram.builder()
> .addRuleInstance(AggregateExpandDistinctAggregatesRule.INSTANCE)
> //.addRuleInstance(AggregateProjectMergeRule.INSTANCE)
> .build();
> checkPlanning(program,
> "select type, count(distinct acctno), sum(distinct balance)"
> + " from customer.account group by type");
> }
> LogicalProject(TYPE=[$0], EXPR$1=[$1], EXPR$2=[CAST($2):INTEGER NOT NULL])
> LogicalAggregate(group=[{0}], EXPR$1=[COUNT($1) FILTER $3], EXPR$2=[SUM($2)
> FILTER $4])
> LogicalProject(TYPE=[$0], ACCTNO=[$1], BALANCE=[$2], $g_1=[=($3, 1)],
> $g_2=[=($3, 2)])
> LogicalAggregate(group=[{0, 1, 2}], groups=[[{0, 1}, {0, 2}]],
> $g=[GROUPING($0, $1, $2)])
> LogicalProject(TYPE=[$1], ACCTNO=[$0], BALANCE=[$2])
> LogicalTableScan(table=[[CATALOG, CUSTOMER, ACCOUNT]])
> {code}
> then *AggregateProjectMergeRule.INSTANCE* is added, it will change the
> sub-plan from
> {code}
> LogicalAggregate(group=[{0, 1, 2}], groups=[[{0, 1}, {0, 2}]],
> $g=[GROUPING($0, $1, $2)])
> LogicalProject(TYPE=[$1], ACCTNO=[$0], BALANCE=[$2])
> {code}
> to
> {code}
> LogicalAggregate(group=[{0, 1, 2}], groups=[[{0, 1}, {1, 2}]],
> $g=[GROUPING($1, $0, $2)])
> {code}
> Notes that the groups was changed from {code} groups=[[{0, 1}, {0, 2}]]
> {code} to
> {code} groups=[[{0, 1}, {1, 2}]] {code}, but the filter values generated by
> *groupValue* in *AggregateExpandDistinctAggregatesRule* are not changed in
> project:
> {code}
> LogicalProject(TYPE=[$0], ACCTNO=[$1], BALANCE=[$2], $g_1=[=($3, 1)],
> $g_2=[=($3, 2)])
> {code}
> {code}
> // filter values before AggregateProjectMergeRule added
> AggregateExpandDistinctAggregatesRule.groupValue(ImmutableBitSet.of(0,1,2),
> ImmutableBitSet.of(0,1)) is 1
> AggregateExpandDistinctAggregatesRule.groupValue(ImmutableBitSet.of(0,1,2),
> ImmutableBitSet.of(0,2)) is 2
> // filter values after AggregateProjectMergeRule added
> AggregateExpandDistinctAggregatesRule.groupValue(ImmutableBitSet.of(1,0,2),
> ImmutableBitSet.of(0,1)) is 1
> AggregateExpandDistinctAggregatesRule.groupValue(ImmutableBitSet.of(1,0,2),
> ImmutableBitSet.of(1,2)) is 4
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (CALCITE-2122) In DateRangeRules, make either TIMESTAMP or DATE literal, according to target type
[
https://issues.apache.org/jira/browse/CALCITE-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde resolved CALCITE-2122.
--
Resolution: Fixed
Fix Version/s: 1.16.0
Fixed in
[2918b8fe|http://git-wip-us.apache.org/repos/asf/calcite/commit/2918b8fe];
thanks for the PR, [~nishantbangarwa]!
> In DateRangeRules, make either TIMESTAMP or DATE literal, according to target
> type
> --
>
> Key: CALCITE-2122
> URL: https://issues.apache.org/jira/browse/CALCITE-2122
> Project: Calcite
> Issue Type: Bug
> Components: core, druid
>Reporter: slim bouguerra
>Assignee: Nishant Bangarwa
> Fix For: 1.16.0
>
>
> The following test at org.apache.calcite.test.DruidAdapterIT
> {code}
> @Test
> public void testCombinationOfValidAndNotValidAndInterval() {
> final String sql = "SELECT COUNT(*) FROM \"foodmart\" "
> + "WHERE \"timestamp\" < CAST('1997-01-02' as TIMESTAMP) AND "
> + "EXTRACT(MONTH FROM \"timestamp\") = 01 AND EXTRACT(YEAR FROM
> \"timestamp\") = 01 ";
> sql(sql, FOODMART)
> .runs();
> }
> {code}
> Leads to
> {code}
> java.lang.RuntimeException: exception while executing [SELECT COUNT(*) FROM
> "foodmart" WHERE "timestamp" < CAST('1997-01-02' as TIMESTAMP) AND
> EXTRACT(MONTH FROM "timestamp") = 01 AND EXTRACT(YEAR FROM "timestamp") = 01 ]
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.runs(CalciteAssert.java:1411)
> at
> org.apache.calcite.test.DruidAdapterIT.testCombinationOfValidAndNotValidAndInterval(DruidAdapterIT.java:3497)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: java.lang.RuntimeException: With materializationsEnabled=false,
> limit=0
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.runs(CalciteAssert.java:1407)
> ... 23 more
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT COUNT(*)
> FROM "foodmart" WHERE "timestamp" < CAST('1997-01-02' as TIMESTAMP) AND
> EXTRACT(MONTH FROM "timestamp") = 01 AND EXTRACT(YEAR FROM "timestamp") = 01
> ": Error while applying rule FilterDateRangeRule, args
> [rel#19:LogicalFilter.NONE.[](input=rel#18:Subset#0.BINDABLE.[],condition=AND(<($0,
> CAST('1997-01-02'):TIMESTAMP(0) NOT NULL), =(EXTRACT(FLAG(MONTH), $0), 1),
> =(EXTRACT(FLAG(YEAR), $0), 1)))]
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:568)
> ... 24 more
> Caused by: java.lang.RuntimeException: Error while applying rule
>
[jira] [Updated] (CALCITE-2122) In DateRangeRules, make either TIMESTAMP or DATE literal, according to target type
[
https://issues.apache.org/jira/browse/CALCITE-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-2122:
-
Summary: In DateRangeRules, make either TIMESTAMP or DATE literal,
according to target type (was: Continuum of DateRangeRules issues.)
> In DateRangeRules, make either TIMESTAMP or DATE literal, according to target
> type
> --
>
> Key: CALCITE-2122
> URL: https://issues.apache.org/jira/browse/CALCITE-2122
> Project: Calcite
> Issue Type: Bug
> Components: core, druid
>Reporter: slim bouguerra
>Assignee: Nishant Bangarwa
>
> The following test at org.apache.calcite.test.DruidAdapterIT
> {code}
> @Test
> public void testCombinationOfValidAndNotValidAndInterval() {
> final String sql = "SELECT COUNT(*) FROM \"foodmart\" "
> + "WHERE \"timestamp\" < CAST('1997-01-02' as TIMESTAMP) AND "
> + "EXTRACT(MONTH FROM \"timestamp\") = 01 AND EXTRACT(YEAR FROM
> \"timestamp\") = 01 ";
> sql(sql, FOODMART)
> .runs();
> }
> {code}
> Leads to
> {code}
> java.lang.RuntimeException: exception while executing [SELECT COUNT(*) FROM
> "foodmart" WHERE "timestamp" < CAST('1997-01-02' as TIMESTAMP) AND
> EXTRACT(MONTH FROM "timestamp") = 01 AND EXTRACT(YEAR FROM "timestamp") = 01 ]
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.runs(CalciteAssert.java:1411)
> at
> org.apache.calcite.test.DruidAdapterIT.testCombinationOfValidAndNotValidAndInterval(DruidAdapterIT.java:3497)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: java.lang.RuntimeException: With materializationsEnabled=false,
> limit=0
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.runs(CalciteAssert.java:1407)
> ... 23 more
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT COUNT(*)
> FROM "foodmart" WHERE "timestamp" < CAST('1997-01-02' as TIMESTAMP) AND
> EXTRACT(MONTH FROM "timestamp") = 01 AND EXTRACT(YEAR FROM "timestamp") = 01
> ": Error while applying rule FilterDateRangeRule, args
> [rel#19:LogicalFilter.NONE.[](input=rel#18:Subset#0.BINDABLE.[],condition=AND(<($0,
> CAST('1997-01-02'):TIMESTAMP(0) NOT NULL), =(EXTRACT(FLAG(MONTH), $0), 1),
> =(EXTRACT(FLAG(YEAR), $0), 1)))]
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:568)
> ... 24 more
> Caused by: java.lang.RuntimeException: Error while applying rule
> FilterDateRangeRule, args
>
[jira] [Commented] (CALCITE-2123) Bug in the Druid Filter Translation when Comparing String Ref to a Constant Number
[
https://issues.apache.org/jira/browse/CALCITE-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317270#comment-16317270
]
slim bouguerra commented on CALCITE-2123:
-
[~julianhyde], thanks for the pointer. Looking at the code I am suspecting that
issue is how we are invoking {code}
org.apache.calcite.sql.type.SqlTypeUtil#canCastFrom{code} at the function
{code} org.apache.calcite.sql.type.SqlTypeFactoryImpl#leastRestrictiveByCast
{code} TBH am not sure what the boolean var {code} boolean coerce{code} means
exactly.
Althought i when i change the code to the following (instead of coerce=false
use true)
{code}
if (SqlTypeUtil.canCastFrom(type, resultType, true)) {
{code}
we get positive resutl for cast from VarChar to Decimal.
Not sure if that is the actual fix.
am still trying to wrap my mind please let me know if am on the wrong track.
> Bug in the Druid Filter Translation when Comparing String Ref to a Constant
> Number
> --
>
> Key: CALCITE-2123
> URL: https://issues.apache.org/jira/browse/CALCITE-2123
> Project: Calcite
> Issue Type: Bug
> Components: druid
>Reporter: slim bouguerra
>Assignee: Julian Hyde
>
> The following query {code} SELECT COUNT(*) FROM \"foodmart\" WHERE
> \"product_id\" = 16.0{code} Translates to a Druid Table Scan with a String to
> String Selector comparison filter.
> instead we need to have a Bound filter that cast the String to number.
> This is what we should expect.
> {code}
> {"type":"bound","dimension":"product_id","lower":"16.0","lowerStrict":false,"upper":"16.0","upperStrict":false,"ordering":"numeric"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2122) Continuum of DateRangeRules issues.
[
https://issues.apache.org/jira/browse/CALCITE-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317071#comment-16317071
]
Julian Hyde commented on CALCITE-2122:
--
I have reviewed, looks good, made a few cosmetic changes and am now running
full tests. Will commit when they pass.
> Continuum of DateRangeRules issues.
> ---
>
> Key: CALCITE-2122
> URL: https://issues.apache.org/jira/browse/CALCITE-2122
> Project: Calcite
> Issue Type: Bug
> Components: core, druid
>Reporter: slim bouguerra
>Assignee: Nishant Bangarwa
>
> The following test at org.apache.calcite.test.DruidAdapterIT
> {code}
> @Test
> public void testCombinationOfValidAndNotValidAndInterval() {
> final String sql = "SELECT COUNT(*) FROM \"foodmart\" "
> + "WHERE \"timestamp\" < CAST('1997-01-02' as TIMESTAMP) AND "
> + "EXTRACT(MONTH FROM \"timestamp\") = 01 AND EXTRACT(YEAR FROM
> \"timestamp\") = 01 ";
> sql(sql, FOODMART)
> .runs();
> }
> {code}
> Leads to
> {code}
> java.lang.RuntimeException: exception while executing [SELECT COUNT(*) FROM
> "foodmart" WHERE "timestamp" < CAST('1997-01-02' as TIMESTAMP) AND
> EXTRACT(MONTH FROM "timestamp") = 01 AND EXTRACT(YEAR FROM "timestamp") = 01 ]
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.runs(CalciteAssert.java:1411)
> at
> org.apache.calcite.test.DruidAdapterIT.testCombinationOfValidAndNotValidAndInterval(DruidAdapterIT.java:3497)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: java.lang.RuntimeException: With materializationsEnabled=false,
> limit=0
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.runs(CalciteAssert.java:1407)
> ... 23 more
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT COUNT(*)
> FROM "foodmart" WHERE "timestamp" < CAST('1997-01-02' as TIMESTAMP) AND
> EXTRACT(MONTH FROM "timestamp") = 01 AND EXTRACT(YEAR FROM "timestamp") = 01
> ": Error while applying rule FilterDateRangeRule, args
> [rel#19:LogicalFilter.NONE.[](input=rel#18:Subset#0.BINDABLE.[],condition=AND(<($0,
> CAST('1997-01-02'):TIMESTAMP(0) NOT NULL), =(EXTRACT(FLAG(MONTH), $0), 1),
> =(EXTRACT(FLAG(YEAR), $0), 1)))]
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:568)
> ... 24 more
> Caused by: java.lang.RuntimeException: Error while applying rule
> FilterDateRangeRule, args
> [rel#19:LogicalFilter.NONE.[](input=rel#18:Subset#0.BINDABLE.[],condition=AND(<($0,
> CAST('1997-01-02'):TIMESTAMP(0) NOT NULL),
[jira] [Commented] (CALCITE-2123) Bug in the Druid Filter Translation when Comparing String Ref to a Constant Number
[
https://issues.apache.org/jira/browse/CALCITE-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316914#comment-16316914
]
Julian Hyde commented on CALCITE-2123:
--
Probably SqlToRelConverter.Blackboard.convertExpression. Also, run any of the
test cases in SqlToRelConverterTest in a debugger, and put a breakpoint in
RexCall's constructor, looking for a call to EQUALS.
> Bug in the Druid Filter Translation when Comparing String Ref to a Constant
> Number
> --
>
> Key: CALCITE-2123
> URL: https://issues.apache.org/jira/browse/CALCITE-2123
> Project: Calcite
> Issue Type: Bug
> Components: druid
>Reporter: slim bouguerra
>Assignee: Julian Hyde
>
> The following query {code} SELECT COUNT(*) FROM \"foodmart\" WHERE
> \"product_id\" = 16.0{code} Translates to a Druid Table Scan with a String to
> String Selector comparison filter.
> instead we need to have a Bound filter that cast the String to number.
> This is what we should expect.
> {code}
> {"type":"bound","dimension":"product_id","lower":"16.0","lowerStrict":false,"upper":"16.0","upperStrict":false,"ordering":"numeric"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2123) Bug in the Druid Filter Translation when Comparing String Ref to a Constant Number
[
https://issues.apache.org/jira/browse/CALCITE-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316898#comment-16316898
]
slim bouguerra commented on CALCITE-2123:
-
[~julianhyde] Thanks, can you point me out to where the RexNode is actually
created out of ATS or Lex Parser?
> Bug in the Druid Filter Translation when Comparing String Ref to a Constant
> Number
> --
>
> Key: CALCITE-2123
> URL: https://issues.apache.org/jira/browse/CALCITE-2123
> Project: Calcite
> Issue Type: Bug
> Components: druid
>Reporter: slim bouguerra
>Assignee: Julian Hyde
>
> The following query {code} SELECT COUNT(*) FROM \"foodmart\" WHERE
> \"product_id\" = 16.0{code} Translates to a Druid Table Scan with a String to
> String Selector comparison filter.
> instead we need to have a Bound filter that cast the String to number.
> This is what we should expect.
> {code}
> {"type":"bound","dimension":"product_id","lower":"16.0","lowerStrict":false,"upper":"16.0","upperStrict":false,"ordering":"numeric"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2123) Bug in the Druid Filter Translation when Comparing String Ref to a Constant Number
[
https://issues.apache.org/jira/browse/CALCITE-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316878#comment-16316878
]
Julian Hyde commented on CALCITE-2123:
--
OK, we are agreed. Let's tighten the type-checking rules for when these
RexNodes are created. There may be some pain, though; I'll do my best to help
you get through that.
> Bug in the Druid Filter Translation when Comparing String Ref to a Constant
> Number
> --
>
> Key: CALCITE-2123
> URL: https://issues.apache.org/jira/browse/CALCITE-2123
> Project: Calcite
> Issue Type: Bug
> Components: druid
>Reporter: slim bouguerra
>Assignee: Julian Hyde
>
> The following query {code} SELECT COUNT(*) FROM \"foodmart\" WHERE
> \"product_id\" = 16.0{code} Translates to a Druid Table Scan with a String to
> String Selector comparison filter.
> instead we need to have a Bound filter that cast the String to number.
> This is what we should expect.
> {code}
> {"type":"bound","dimension":"product_id","lower":"16.0","lowerStrict":false,"upper":"16.0","upperStrict":false,"ordering":"numeric"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2123) Bug in the Druid Filter Translation when Comparing String Ref to a Constant Number
[
https://issues.apache.org/jira/browse/CALCITE-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316870#comment-16316870
]
slim bouguerra commented on CALCITE-2123:
-
Currently calcite does not add the extra CAST (looking at debugger) The filter
is {code} =($1, 16.0) {code} And the type of input ref is VarChar. Agree we
need to have the implicit cast to make it more consistent.
> Bug in the Druid Filter Translation when Comparing String Ref to a Constant
> Number
> --
>
> Key: CALCITE-2123
> URL: https://issues.apache.org/jira/browse/CALCITE-2123
> Project: Calcite
> Issue Type: Bug
> Components: druid
>Reporter: slim bouguerra
>Assignee: Julian Hyde
>
> The following query {code} SELECT COUNT(*) FROM \"foodmart\" WHERE
> \"product_id\" = 16.0{code} Translates to a Druid Table Scan with a String to
> String Selector comparison filter.
> instead we need to have a Bound filter that cast the String to number.
> This is what we should expect.
> {code}
> {"type":"bound","dimension":"product_id","lower":"16.0","lowerStrict":false,"upper":"16.0","upperStrict":false,"ordering":"numeric"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2123) Bug in the Druid Filter Translation when Comparing String Ref to a Constant Number
[
https://issues.apache.org/jira/browse/CALCITE-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316857#comment-16316857
]
Julian Hyde commented on CALCITE-2123:
--
The thing this has in common with CALCITE-2122 is implicit casts. {{WHERE
\"product_id\" = 16.0}} should be translated to the same RexNode tree as
{{WHERE CAST(\"product_id\" AS DECIMAL) = 16.0}}. Are we actually creating
those CAST calls in Rex? I know those Rex calls make it harder to push down
expressions, but without them we will hit inconsistencies during translation.
> Bug in the Druid Filter Translation when Comparing String Ref to a Constant
> Number
> --
>
> Key: CALCITE-2123
> URL: https://issues.apache.org/jira/browse/CALCITE-2123
> Project: Calcite
> Issue Type: Bug
> Components: druid
>Reporter: slim bouguerra
>Assignee: Julian Hyde
>
> The following query {code} SELECT COUNT(*) FROM \"foodmart\" WHERE
> \"product_id\" = 16.0{code} Translates to a Druid Table Scan with a String to
> String Selector comparison filter.
> instead we need to have a Bound filter that cast the String to number.
> This is what we should expect.
> {code}
> {"type":"bound","dimension":"product_id","lower":"16.0","lowerStrict":false,"upper":"16.0","upperStrict":false,"ordering":"numeric"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2124) aggCall type mismatch error in AggregateExpandDistinctAggregatesRule
[
https://issues.apache.org/jira/browse/CALCITE-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316842#comment-16316842
]
Julian Hyde commented on CALCITE-2124:
--
Probably due to AggregateExpandDistinctAggregatesRule forgetting the fact that
MAX applied to a NOT NULL column is NULL only in one circumstance -- when there
are zero input rows, i.e. on {{GROUP BY ()}} or no {{GROUP BY}}.
> aggCall type mismatch error in AggregateExpandDistinctAggregatesRule
>
>
> Key: CALCITE-2124
> URL: https://issues.apache.org/jira/browse/CALCITE-2124
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.15.0
>Reporter: godfrey he
>Assignee: Julian Hyde
>
> {code}
> @Test public void testDistinctCountWithoutGroupBy() {
> final HepProgram program = HepProgram.builder()
> .addRuleInstance(AggregateExpandDistinctAggregatesRule.INSTANCE)
> .addRuleInstance(AggregateProjectMergeRule.INSTANCE)
> .build();
> checkPlanning(program,
> "select max(deptno), count(distinct ename) from sales.emp");
> }
> {code}
> exception:
> {code}
> java.lang.AssertionError: type mismatch:
> aggCall type:
> INTEGER
> inferred type:
> INTEGER NOT NULL
> at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31)
> at org.apache.calcite.plan.RelOptUtil.eq(RelOptUtil.java:1838)
> at
> org.apache.calcite.rel.core.Aggregate.typeMatchesInferred(Aggregate.java:428)
> at org.apache.calcite.rel.core.Aggregate.(Aggregate.java:161)
> at
> org.apache.calcite.rel.logical.LogicalAggregate.(LogicalAggregate.java:65)
> at
> org.apache.calcite.rel.logical.LogicalAggregate.create_(LogicalAggregate.java:110)
> at
> org.apache.calcite.rel.logical.LogicalAggregate.create(LogicalAggregate.java:100)
> at
> org.apache.calcite.rel.core.RelFactories$AggregateFactoryImpl.createAggregate(RelFactories.java:213)
> at org.apache.calcite.tools.RelBuilder.aggregate(RelBuilder.java:1268)
> at org.apache.calcite.tools.RelBuilder.aggregate(RelBuilder.java:1840)
> at
> org.apache.calcite.rel.rules.AggregateExpandDistinctAggregatesRule.rewriteUsingGroupingSets(AggregateExpandDistinctAggregatesRule.java:413)
> at
> org.apache.calcite.rel.rules.AggregateExpandDistinctAggregatesRule.onMatch(AggregateExpandDistinctAggregatesRule.java:172)
> at
> org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:317)
> at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:508)
> at
> org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:387)
> at
> org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:252)
> at
> org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127)
> at
> org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:211)
> at
> org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:198)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:165)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:120)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:95)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:84)
> at
> org.apache.calcite.test.RelOptRulesTest.testDistinctCountWithoutGroupBy(RelOptRulesTest.java:761)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2125) error plan result that using AggregateExpandDistinctAggregatesRule.INSTANCE and AggregateProjectMergeRule.INSTANCE
[
https://issues.apache.org/jira/browse/CALCITE-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316814#comment-16316814
]
Julian Hyde commented on CALCITE-2125:
--
It does indeed look like a bug. It seems that that the bug is in
AggregateProjectMergeRule; if so, you could use
AggregateExpandDistinctAggregatesRule in the pre-program in RelOptRulesTest, so
that the test is testing only what AggregateProjectMergeRule does.
> error plan result that using AggregateExpandDistinctAggregatesRule.INSTANCE
> and AggregateProjectMergeRule.INSTANCE
> ---
>
> Key: CALCITE-2125
> URL: https://issues.apache.org/jira/browse/CALCITE-2125
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.15.0
>Reporter: godfrey he
>Assignee: Julian Hyde
>
> test case:
> {code}
> @Test public void testDistinctCount0() {
> final HepProgram program = HepProgram.builder()
> .addRuleInstance(AggregateExpandDistinctAggregatesRule.INSTANCE)
> .addRuleInstance(AggregateProjectMergeRule.INSTANCE)
> .build();
> checkPlanning(program,
> "select type, count(distinct acctno), sum(distinct balance)"
> + " from customer.account group by type");
> }
> {code}
> current result:
> {code}
>
>
>
>
>
>
>
>
>
>
>
> {code}
> However, the result plan is wrong.
> first, if we only use *AggregateExpandDistinctAggregatesRule.INSTANCE* to
> optimize the query, the result plan is correct:
> {code}
> @Test public void testDistinctCount0() {
> final HepProgram program = HepProgram.builder()
> .addRuleInstance(AggregateExpandDistinctAggregatesRule.INSTANCE)
> //.addRuleInstance(AggregateProjectMergeRule.INSTANCE)
> .build();
> checkPlanning(program,
> "select type, count(distinct acctno), sum(distinct balance)"
> + " from customer.account group by type");
> }
> LogicalProject(TYPE=[$0], EXPR$1=[$1], EXPR$2=[CAST($2):INTEGER NOT NULL])
> LogicalAggregate(group=[{0}], EXPR$1=[COUNT($1) FILTER $3], EXPR$2=[SUM($2)
> FILTER $4])
> LogicalProject(TYPE=[$0], ACCTNO=[$1], BALANCE=[$2], $g_1=[=($3, 1)],
> $g_2=[=($3, 2)])
> LogicalAggregate(group=[{0, 1, 2}], groups=[[{0, 1}, {0, 2}]],
> $g=[GROUPING($0, $1, $2)])
> LogicalProject(TYPE=[$1], ACCTNO=[$0], BALANCE=[$2])
> LogicalTableScan(table=[[CATALOG, CUSTOMER, ACCOUNT]])
> {code}
> then *AggregateProjectMergeRule.INSTANCE* is added, it will change the
> sub-plan from
> {code}
> LogicalAggregate(group=[{0, 1, 2}], groups=[[{0, 1}, {0, 2}]],
> $g=[GROUPING($0, $1, $2)])
> LogicalProject(TYPE=[$1], ACCTNO=[$0], BALANCE=[$2])
> {code}
> to
> {code}
> LogicalAggregate(group=[{0, 1, 2}], groups=[[{0, 1}, {1, 2}]],
> $g=[GROUPING($1, $0, $2)])
> {code}
> Notes that the groups was changed from {code} groups=[[{0, 1}, {0, 2}]]
> {code} to
> {code} groups=[[{0, 1}, {1, 2}]] {code}, but the filter values generated by
> *groupValue* in *AggregateExpandDistinctAggregatesRule* are not changed in
> project:
> {code}
> LogicalProject(TYPE=[$0], ACCTNO=[$1], BALANCE=[$2], $g_1=[=($3, 1)],
> $g_2=[=($3, 2)])
> {code}
> {code}
> // filter values before AggregateProjectMergeRule added
> AggregateExpandDistinctAggregatesRule.groupValue(ImmutableBitSet.of(0,1,2),
> ImmutableBitSet.of(0,1)) is 1
> AggregateExpandDistinctAggregatesRule.groupValue(ImmutableBitSet.of(0,1,2),
> ImmutableBitSet.of(0,2)) is 2
> // filter values after AggregateProjectMergeRule added
> AggregateExpandDistinctAggregatesRule.groupValue(ImmutableBitSet.of(1,0,2),
> ImmutableBitSet.of(0,1)) is 1
> AggregateExpandDistinctAggregatesRule.groupValue(ImmutableBitSet.of(1,0,2),
> ImmutableBitSet.of(1,2)) is 4
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2009) Possible bug in interpreting ( IN ) OR ( IN ) logic
[
https://issues.apache.org/jira/browse/CALCITE-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316151#comment-16316151
]
Piotr Bojko commented on CALCITE-2009:
--
Will try in CALCITE-2088
> Possible bug in interpreting ( IN ) OR ( IN ) logic
> -
>
> Key: CALCITE-2009
> URL: https://issues.apache.org/jira/browse/CALCITE-2009
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.14.0, 1.13.0, 1.15.0
>Reporter: Piotr Bojko
>Assignee: Julian Hyde
> Fix For: 1.16.0
>
> Attachments: logs-when-or-is-used.log, logs-when-union-is-used.log
>
>
> I have schema
> {code:javascript}
> {
> "version": "1.0",
> "defaultSchema": "JIRA",
> "schemas": [
> {
> "name": "INTERNAL",
> "type": "custom",
> "factory": "%%UNDERLYING_SCHEMA_FACTORY",
> "operand": {}
> }, {
> "name": "JIRA",
> "type": "custom",
> "factory":
> "org.apache.calcite.schema.impl.AbstractSchema$Factory",
> "operand": {},
> "tables": [
> {
> "name": "ISSUES",
> "type": "view",
> "sql": [
> "SELECT JI.ID, JI.SUMMARY,
> JI.PROJECT ",
> "FROM INTERNAL.JIRAISSUE as JI
> ",
> "WHERE JI.ID IN (SELECT EAI.ID
> FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ",
> "UNION ",
> "SELECT JI.ID, JI.SUMMARY,
> JI.PROJECT ",
> "FROM INTERNAL.JIRAISSUE as JI
> ",
> "WHERE JI.PROJECT IN (SELECT
> AP.ID FROM ALLOWED_PROJECTS AS AP)"
> ]
> },
> {
> "name": "ALLOWED_PROJECTS",
> "type": "table",
> "factory": "%%DELEGATING_TABLE_FACTORY"
> },
> {
> "name": "EXPLICIT_ALLOWED_ISSUES",
> "type": "table",
> "factory": "%%DELEGATING_TABLE_FACTORY"
> }
> ]
> }
> ]
> }
> {code}
> Where INTERNAL schema points to JDBC native JIRA Schema (through my custom
> factory, but it only wraps passing datasource instead of user/password/url to
> db) and JIRA schema has one view and two tables (factories generate java data
> wrapped in AbstractQueryableTable)
> When running "SELECT * FROM ISSUES" all works.
> But when changing the view from UNION construction to OR - engine provides no
> results:
> {code:javascript}
> {
> "version": "1.0",
> "defaultSchema": "JIRA",
> "schemas": [
> {
> "name": "INTERNAL",
> "type": "custom",
> "factory": "%%UNDERLYING_SCHEMA_FACTORY",
> "operand": {}
> }, {
> "name": "JIRA",
> "type": "custom",
> "factory":
> "org.apache.calcite.schema.impl.AbstractSchema$Factory",
> "operand": {},
> "tables": [
> {
> "name": "ISSUES",
> "type": "view",
> "sql": [
> "SELECT JI.ID, JI.SUMMARY,
> JI.PROJECT ",
> "FROM INTERNAL.JIRAISSUE as JI
> ",
> "WHERE JI.ID IN (SELECT EAI.ID
> FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ",
> "OR (JI.PROJECT IN (SELECT
> AP.ID FROM ALLOWED_PROJECTS AS AP))"
> ]
> },
> {
> "name": "ALLOWED_PROJECTS",
> "type": "table",
> "factory": "%%DELEGATING_TABLE_FACTORY"
> },
> {
> "name": "EXPLICIT_ALLOWED_ISSUES",
