[jira] [Comment Edited] (CALCITE-1581) UDTF like in hive
[
https://issues.apache.org/jira/browse/CALCITE-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804052#comment-16804052
]
pengzhiwei edited comment on CALCITE-1581 at 3/29/19 5:54 AM:
--
Hi [~julianhyde], I have make a pr
[https://github.com/apache/calcite/pull/1138] for this issue.
In this pr,I mainly do the follow things:
* Rewrite the "parser.jj" file to support "udtf() as (f0,f1)" in the sql
parser stage with minimum modification.
* add allowSelectTableFunction() in SqlConformance and also add a kind of Hive
SqlConformance.
* In SqlValidator stage, I rewrite the "udtf() as (f0,f1)" to the
lateral-table sql-node. With this rewritting,there is nothing need to do in the
SqlToRelConverter.
was (Author: pzw2018):
Hi [~julianhyde], I have make a pr
[https://github.com/apache/calcite/pull/1138] for this issue.
In this pr,I mainly do the follow things:
* Rewrite the "parser.jj" file to support "udtf() as (f0,f1)" in the sql
parser stage with minimum modification.
* add allowSelectTableFunction() in SqlConformance and also add a kind of Hive
SqlConformance.
* In SqlValidator stage, I rewrite the "udtf() as (f0,f1)" to the
lateral-table sql-node. With this rewritting,this is nothing need to do in the
SqlToRelConverter.
> UDTF like in hive
> -
>
> Key: CALCITE-1581
> URL: https://issues.apache.org/jira/browse/CALCITE-1581
> Project: Calcite
> Issue Type: New Feature
>Reporter: Xiaoyong Deng
>Assignee: Sihua Zhou
>Priority: Major
> Labels: pull-request-available, udtf
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Support one row in and multi-column/multi-row out(one-to-many mapping), just
> like udtf in hive.
> The query would like this:
> {code}
> select
> func(c0, c1) as (f0, f1, f2)
> from table_name;
> {code}
> c0 and c1 are 'table_name' columns. f0, f1 and f2 are new generated columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2960) CalciteCatalogReader use a hard-coding config to get functions
[
https://issues.apache.org/jira/browse/CALCITE-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804578#comment-16804578
]
Danny Chan commented on CALCITE-2960:
-
Got your idea, it's weird that Calcite itself never use CalciteCatalogReader as
a SqlOperatorTable lookup, the only usage is for [1] which is used in a test
case. Usually we will use a StdSqlOperatorTable chained with a
ListSqlOperatorTable.
But your usage is ok, after all, CalciteCatalogReader is definitely a
SqlOperatorTable. Within CalciteCatalogReader, functions are managed by
CalciteSchema, which are in the scope of a Catalog.
[1]
https://github.com/apache/calcite/blob/0537f27d87598dea888050a4cc63cf4d6fe027d7/core/src/main/java/org/apache/calcite/prepare/CalciteCatalogReader.java#L276
> CalciteCatalogReader use a hard-coding config to get functions
> --
>
> Key: CALCITE-2960
> URL: https://issues.apache.org/jira/browse/CALCITE-2960
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: Lai Zhou
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> CalciteCatalogReader use a hard-coding config to get functions
> {code:java}
> functions2.addAll(schema.getFunctions(name, true));
> {code}
> the right way is：
> {code:java}
> functions2.addAll(schema.getFunctions(name, config.caseSensitive()));
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2960) CalciteCatalogReader use a hard-coding config to get functions
[
https://issues.apache.org/jira/browse/CALCITE-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804563#comment-16804563
]
Lai Zhou commented on CALCITE-2960:
---
It's my code.
{code:java}
protected FrameworkConfig createFrameworkConfig(
SqlParser.Config sqlParserConfig, SqlOperatorTable sqlOperatorTable,
CalciteConnectionConfig calciteConnectionConfig) {
return Frameworks
.newConfigBuilder()
.defaultSchema(rootSchema)
.parserConfig(sqlParserConfig)
.operatorTable(sqlOperatorTable)
.context(Contexts.of(calciteConnectionConfig))
.build();
}
{code}
I just use a customized ChainedSqlOperatorTable that includes a
CalciteCatalogReader, the CalciteCatalogReader
holds the rootSchema I created. So I can register a function into the
rootSchema, when executing a sql query,
the planner will look up an operator from the CalciteCatalogReader .
Part of my code for parsing and converting the sql query is like:
{code:java}
try (Planner planner = Frameworks.getPlanner(frameworkConfig)) {
RelRoot root;
final SqlNode parsedSqlNode = planner.parse(sql);
final Pair validatedSqlNodeAndType = planner
.validateAndGetType(
parsedSqlNode);
root = planner.rel(validatedSqlNodeAndType.getKey());
...{code}
> CalciteCatalogReader use a hard-coding config to get functions
> --
>
> Key: CALCITE-2960
> URL: https://issues.apache.org/jira/browse/CALCITE-2960
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: Lai Zhou
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> CalciteCatalogReader use a hard-coding config to get functions
> {code:java}
> functions2.addAll(schema.getFunctions(name, true));
> {code}
> the right way is：
> {code:java}
> functions2.addAll(schema.getFunctions(name, config.caseSensitive()));
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2960) CalciteCatalogReader use a hard-coding config to get functions
[
https://issues.apache.org/jira/browse/CALCITE-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804563#comment-16804563
]
Lai Zhou edited comment on CALCITE-2960 at 3/29/19 3:53 AM:
It's my code.
{code:java}
protected FrameworkConfig createFrameworkConfig(
SqlParser.Config sqlParserConfig, SqlOperatorTable sqlOperatorTable,
CalciteConnectionConfig calciteConnectionConfig) {
return Frameworks
.newConfigBuilder()
.defaultSchema(rootSchema)
.parserConfig(sqlParserConfig)
.operatorTable(sqlOperatorTable)
.context(Contexts.of(calciteConnectionConfig))
.build();
}
{code}
I just use a customized ChainedSqlOperatorTable that includes a
CalciteCatalogReader, and the CalciteCatalogReader
holds the rootSchema I created. So I can register a function into the
rootSchema, when executing a sql query,
the planner will look up an operator from the CalciteCatalogReader .
Part of my code for parsing and converting the sql query is like:
{code:java}
try (Planner planner = Frameworks.getPlanner(frameworkConfig)) {
RelRoot root;
final SqlNode parsedSqlNode = planner.parse(sql);
final Pair validatedSqlNodeAndType = planner
.validateAndGetType(
parsedSqlNode);
root = planner.rel(validatedSqlNodeAndType.getKey());
...{code}
was (Author: hhlai1990):
It's my code.
{code:java}
protected FrameworkConfig createFrameworkConfig(
SqlParser.Config sqlParserConfig, SqlOperatorTable sqlOperatorTable,
CalciteConnectionConfig calciteConnectionConfig) {
return Frameworks
.newConfigBuilder()
.defaultSchema(rootSchema)
.parserConfig(sqlParserConfig)
.operatorTable(sqlOperatorTable)
.context(Contexts.of(calciteConnectionConfig))
.build();
}
{code}
I just use a customized ChainedSqlOperatorTable that includes a
CalciteCatalogReader, the CalciteCatalogReader
holds the rootSchema I created. So I can register a function into the
rootSchema, when executing a sql query,
the planner will look up an operator from the CalciteCatalogReader .
Part of my code for parsing and converting the sql query is like:
{code:java}
try (Planner planner = Frameworks.getPlanner(frameworkConfig)) {
RelRoot root;
final SqlNode parsedSqlNode = planner.parse(sql);
final Pair validatedSqlNodeAndType = planner
.validateAndGetType(
parsedSqlNode);
root = planner.rel(validatedSqlNodeAndType.getKey());
...{code}
> CalciteCatalogReader use a hard-coding config to get functions
> --
>
> Key: CALCITE-2960
> URL: https://issues.apache.org/jira/browse/CALCITE-2960
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: Lai Zhou
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> CalciteCatalogReader use a hard-coding config to get functions
> {code:java}
> functions2.addAll(schema.getFunctions(name, true));
> {code}
> the right way is：
> {code:java}
> functions2.addAll(schema.getFunctions(name, config.caseSensitive()));
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2921) nullif(null,y) throws exception in verification
[
https://issues.apache.org/jira/browse/CALCITE-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804546#comment-16804546
]
pengzhiwei edited comment on CALCITE-2921 at 3/29/19 3:07 AM:
--
Thanks [~julianhyde], I will continue the work.
was (Author: pzw2018):
Thanks [~julianhyde], I will continue work.
> nullif(null,y) throws exception in verification
>
>
> Key: CALCITE-2921
> URL: https://issues.apache.org/jira/browse/CALCITE-2921
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently calcite will translate "nullif(x,y)" to a case-when expression,just
> like "case when x = y then null else x".So when "x" is null literal,a
> exception throws out as follow:
> {code:java}
> ELSE clause or at least one THEN clause must be non-NULL
> {code}
> I have test in mysql,"nullif(null,y) works well.So I think we should allow
> this usage of "nullif".
> There are two ways to fix this issue:
> 1) Skip the check for "foundNotNull" in SqlCaseOperator#checkOperandTypes:
>
> {code:java}
> if (!foundNotNull) {
> // according to the sql standard we can not have all of the THEN
> // statements and the ELSE returning null
> if (throwOnFailure) {
> throw callBinding.newError(RESOURCE.mustNotNullInElse());
> }
> return false;
> }{code}
> However, as the comment says, we cannot have all of the THEN and ELSE
> returning null.
> 2) Disable the translation from nullif to case-when and keep "nullif" as it
> is.
> Any suggestion is welcomed,Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2921) nullif(null,y) throws exception in verification
[
https://issues.apache.org/jira/browse/CALCITE-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804546#comment-16804546
]
pengzhiwei commented on CALCITE-2921:
-
Thanks [~julianhyde], I will continue work.
> nullif(null,y) throws exception in verification
>
>
> Key: CALCITE-2921
> URL: https://issues.apache.org/jira/browse/CALCITE-2921
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently calcite will translate "nullif(x,y)" to a case-when expression,just
> like "case when x = y then null else x".So when "x" is null literal,a
> exception throws out as follow:
> {code:java}
> ELSE clause or at least one THEN clause must be non-NULL
> {code}
> I have test in mysql,"nullif(null,y) works well.So I think we should allow
> this usage of "nullif".
> There are two ways to fix this issue:
> 1) Skip the check for "foundNotNull" in SqlCaseOperator#checkOperandTypes:
>
> {code:java}
> if (!foundNotNull) {
> // according to the sql standard we can not have all of the THEN
> // statements and the ELSE returning null
> if (throwOnFailure) {
> throw callBinding.newError(RESOURCE.mustNotNullInElse());
> }
> return false;
> }{code}
> However, as the comment says, we cannot have all of the THEN and ELSE
> returning null.
> 2) Disable the translation from nullif to case-when and keep "nullif" as it
> is.
> Any suggestion is welcomed,Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (CALCITE-2744) Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query with a multi-param aggregate function in subquery
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan closed CALCITE-2744.
---
Resolution: Won't Fix
> Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query
> with a multi-param aggregate function in subquery
> --
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.17.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at
[jira] [Updated] (CALCITE-2744) Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query with a multi-param aggregate function in subquery
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-2744:
Fix Version/s: (was: 1.20.0)
> Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query
> with a multi-param aggregate function in subquery
> --
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.17.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at
[jira] [Commented] (CALCITE-2744) Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query with a multi-param aggregate function in subquery
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804543#comment-16804543
]
Danny Chan commented on CALCITE-2744:
-
It seems that it is an edge case that an agg function can take 2 parameters,
fix of just 2 is not that necessary, after all there are maybe 3 or more
parameters, so i kind of not fix this and just let it throw exception.
> Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query
> with a multi-param aggregate function in subquery
> --
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.17.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at
[jira] [Updated] (CALCITE-2744) Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query with a multi-param aggregate function in subquery
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-2744:
Fix Version/s: 1.20.0
> Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query
> with a multi-param aggregate function in subquery
> --
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.17.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.20.0
>
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at
[jira] [Assigned] (CALCITE-2744) Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query with a multi-param aggregate function in subquery
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan reassigned CALCITE-2744:
---
Assignee: Danny Chan
> Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query
> with a multi-param aggregate function in subquery
> --
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.17.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>
[jira] [Comment Edited] (CALCITE-2960) CalciteCatalogReader use a hard-coding config to get functions
[
https://issues.apache.org/jira/browse/CALCITE-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804519#comment-16804519
]
Lai Zhou edited comment on CALCITE-2960 at 3/29/19 1:59 AM:
[~danny0405], it is really a part of issue 2928.
In my case ,I use the FrameworkConfig to create a sql planner，the config is
like:
{code:java}
Frameworks
.newConfigBuilder()
.defaultSchema(rootSchema)
.parserConfig(sqlParserConfig)
.operatorTable(sqlOperatorTable)
.context(Contexts.of(calciteConnectionConfig))
.build();
{code}
User-defined functions will be added into the rootSchema , and a
CalciteCatalogReader will be created based on this rootSchema.
{code:java}
new CalciteCatalogReader( CalciteSchema.from(rootSchema),
CalciteSchema.from(rootSchema).path(null), new JavaTypeFactoryImpl(typeSystem),
calciteConnectionConfig);
{code}
Then the FrameworkConfig will add the CalciteCatalogReader to it's
ChainedSqlOperatorTable.
{code:java}
createFrameworkConfig(sqlParserConfig,
ChainedSqlOperatorTable.of(sqlStdOperatorTable,
calciteCatalogReader), calciteConnectionConfig);
{code}
So in the runtime , the sql planner will look up operators from the
CalciteCatalogReader .
was (Author: hhlai1990):
[~danny0405], it is really a part of issue 2928.
In my case ,I use the FrameworkConfig to create a sql planner，the config is
like:
{code:java}
Frameworks
.newConfigBuilder()
.defaultSchema(rootSchema)
.parserConfig(sqlParserConfig)
.operatorTable(sqlStdOperatorTable)
.context(Contexts.of(calciteConnectionConfig))
.build();
{code}
User-defined functions will be added into the rootSchema , and a
CalciteCatalogReader will be created based on this rootSchema.
{code:java}
new CalciteCatalogReader( CalciteSchema.from(rootSchema),
CalciteSchema.from(rootSchema).path(null), new JavaTypeFactoryImpl(typeSystem),
calciteConnectionConfig);
{code}
Then the FrameworkConfig will add the CalciteCatalogReader to it's
ChainedSqlOperatorTable.
{code:java}
createFrameworkConfig(sqlParserConfig,
ChainedSqlOperatorTable.of(sqlStdOperatorTable,
calciteCatalogReader), calciteConnectionConfig);
{code}
So in the runtime , the sql planner will look up operators from the
CalciteCatalogReader .
> CalciteCatalogReader use a hard-coding config to get functions
> --
>
> Key: CALCITE-2960
> URL: https://issues.apache.org/jira/browse/CALCITE-2960
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: Lai Zhou
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> CalciteCatalogReader use a hard-coding config to get functions
> {code:java}
> functions2.addAll(schema.getFunctions(name, true));
> {code}
> the right way is：
> {code:java}
> functions2.addAll(schema.getFunctions(name, config.caseSensitive()));
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2960) CalciteCatalogReader use a hard-coding config to get functions
[
https://issues.apache.org/jira/browse/CALCITE-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804519#comment-16804519
]
Lai Zhou commented on CALCITE-2960:
---
[~danny0405], it is really a part of issue 2928.
In my case ,I use the FrameworkConfig to create a sql planner，the config is
like:
{code:java}
Frameworks
.newConfigBuilder()
.defaultSchema(rootSchema)
.parserConfig(sqlParserConfig)
.operatorTable(sqlStdOperatorTable)
.context(Contexts.of(calciteConnectionConfig))
.build();
{code}
User-defined functions will be added into the rootSchema , and a
CalciteCatalogReader will be created based on this rootSchema.
{code:java}
new CalciteCatalogReader( CalciteSchema.from(rootSchema),
CalciteSchema.from(rootSchema).path(null), new JavaTypeFactoryImpl(typeSystem),
calciteConnectionConfig);
{code}
Then the FrameworkConfig will add the CalciteCatalogReader to it's
ChainedSqlOperatorTable.
{code:java}
createFrameworkConfig(sqlParserConfig,
ChainedSqlOperatorTable.of(sqlStdOperatorTable,
calciteCatalogReader), calciteConnectionConfig);
{code}
So in the runtime , the sql planner will look up operators from the
CalciteCatalogReader .
> CalciteCatalogReader use a hard-coding config to get functions
> --
>
> Key: CALCITE-2960
> URL: https://issues.apache.org/jira/browse/CALCITE-2960
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: Lai Zhou
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> CalciteCatalogReader use a hard-coding config to get functions
> {code:java}
> functions2.addAll(schema.getFunctions(name, true));
> {code}
> the right way is：
> {code:java}
> functions2.addAll(schema.getFunctions(name, config.caseSensitive()));
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2960) CalciteCatalogReader use a hard-coding config to get functions
[
https://issues.apache.org/jira/browse/CALCITE-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804519#comment-16804519
]
Lai Zhou edited comment on CALCITE-2960 at 3/29/19 1:57 AM:
[~danny0405], it is really a part of issue 2928.
In my case ,I use the FrameworkConfig to create a sql planner，the config is
like:
{code:java}
Frameworks
.newConfigBuilder()
.defaultSchema(rootSchema)
.parserConfig(sqlParserConfig)
.operatorTable(sqlStdOperatorTable)
.context(Contexts.of(calciteConnectionConfig))
.build();
{code}
User-defined functions will be added into the rootSchema , and a
CalciteCatalogReader will be created based on this rootSchema.
{code:java}
new CalciteCatalogReader( CalciteSchema.from(rootSchema),
CalciteSchema.from(rootSchema).path(null), new JavaTypeFactoryImpl(typeSystem),
calciteConnectionConfig);
{code}
Then the FrameworkConfig will add the CalciteCatalogReader to it's
ChainedSqlOperatorTable.
{code:java}
createFrameworkConfig(sqlParserConfig,
ChainedSqlOperatorTable.of(sqlStdOperatorTable,
calciteCatalogReader), calciteConnectionConfig);
{code}
So in the runtime , the sql planner will look up operators from the
CalciteCatalogReader .
was (Author: hhlai1990):
[~danny0405], it is really a part of issue 2928.
In my case ,I use the FrameworkConfig to create a sql planner，the config is
like:
{code:java}
Frameworks
.newConfigBuilder()
.defaultSchema(rootSchema)
.parserConfig(sqlParserConfig)
.operatorTable(sqlStdOperatorTable)
.context(Contexts.of(calciteConnectionConfig))
.build();
{code}
User-defined functions will be added into the rootSchema , and a
CalciteCatalogReader will be created based on this rootSchema.
{code:java}
new CalciteCatalogReader( CalciteSchema.from(rootSchema),
CalciteSchema.from(rootSchema).path(null), new JavaTypeFactoryImpl(typeSystem),
calciteConnectionConfig);
{code}
Then the FrameworkConfig will add the CalciteCatalogReader to it's
ChainedSqlOperatorTable.
{code:java}
createFrameworkConfig(sqlParserConfig,
ChainedSqlOperatorTable.of(sqlStdOperatorTable,
calciteCatalogReader), calciteConnectionConfig);
{code}
So in the runtime , the sql planner will look up operators from the
CalciteCatalogReader .
> CalciteCatalogReader use a hard-coding config to get functions
> --
>
> Key: CALCITE-2960
> URL: https://issues.apache.org/jira/browse/CALCITE-2960
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: Lai Zhou
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> CalciteCatalogReader use a hard-coding config to get functions
> {code:java}
> functions2.addAll(schema.getFunctions(name, true));
> {code}
> the right way is：
> {code:java}
> functions2.addAll(schema.getFunctions(name, config.caseSensitive()));
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-1515) Support TableFunctionScan in RelBuilder
[ https://issues.apache.org/jira/browse/CALCITE-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804513#comment-16804513 ] Chunwei Lei commented on CALCITE-1515: -- [~julianhyde] +1, thank you for your review and fix-up. > Support TableFunctionScan in RelBuilder > --- > > Key: CALCITE-1515 > URL: https://issues.apache.org/jira/browse/CALCITE-1515 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.10.0 >Reporter: Anton Mushin >Assignee: Chunwei Lei >Priority: Minor > Labels: pull-request-available > Time Spent: 4.5h > Remaining Estimate: 0h > > Calcite has a TableFunctionScan, which is a RelNode, takes a function, 0 or > more RelNode inputs, and 0 or more other arguments. > RelBuilder does not support TableFunctionScan yet. > add In RelBuilder TableFunctionScanFactory which will create > LogicalTableFunctionScan like as for LogicalTableScan and factory for others > RelNodes [ > [example|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/tools/RelBuilder.java#L125] > ] > also adding a functionScan method, analogous to the scan method but for table > functions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-2658) Introducing more ReduceExpressionRules
[ https://issues.apache.org/jira/browse/CALCITE-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan resolved CALCITE-2658. Resolution: Fixed Fix Version/s: 1.20.0 > Introducing more ReduceExpressionRules > -- > > Key: CALCITE-2658 > URL: https://issues.apache.org/jira/browse/CALCITE-2658 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Chunwei Lei >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > It is useful to have rules reducing Exchange/Sort/SortExchange keys, e.g., > SELECT key,value FROM (SELECT 1 AS key, value FROM src) r DISTRIBUTE BY key; > can be reduced to > SELECT 1 AS key, value FROM src;# Since singleton requirement may already > required by SELECT. > SELECT key,value FROM (SELECT 1 AS key, value FROM src) r ORDER BY key; > can be reduced to > SELECT 1 AS key, value FROM src;# Since ordering on constant is useless. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2921) nullif(null,y) throws exception in verification
[
https://issues.apache.org/jira/browse/CALCITE-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804256#comment-16804256
]
Julian Hyde commented on CALCITE-2921:
--
I have reviewed PR 1117:
* Could you use java.lang.Void rather than adding a new class Null?
* Can you enable this behavior with a new method
{{SqlConformance.allowNakedNull()}}
* Also add a test for {{nullif(null,y) }} since this is the title of this bug
* Change {{INSTANCE_COUNT}} to a {{ThreadLocal}} since the
function might be used by several tests in parallel
> nullif(null,y) throws exception in verification
>
>
> Key: CALCITE-2921
> URL: https://issues.apache.org/jira/browse/CALCITE-2921
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently calcite will translate "nullif(x,y)" to a case-when expression,just
> like "case when x = y then null else x".So when "x" is null literal,a
> exception throws out as follow:
> {code:java}
> ELSE clause or at least one THEN clause must be non-NULL
> {code}
> I have test in mysql,"nullif(null,y) works well.So I think we should allow
> this usage of "nullif".
> There are two ways to fix this issue:
> 1) Skip the check for "foundNotNull" in SqlCaseOperator#checkOperandTypes:
>
> {code:java}
> if (!foundNotNull) {
> // according to the sql standard we can not have all of the THEN
> // statements and the ELSE returning null
> if (throwOnFailure) {
> throw callBinding.newError(RESOURCE.mustNotNullInElse());
> }
> return false;
> }{code}
> However, as the comment says, we cannot have all of the THEN and ELSE
> returning null.
> 2) Disable the translation from nullif to case-when and keep "nullif" as it
> is.
> Any suggestion is welcomed,Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-1515) Support TableFunctionScan in RelBuilder
[ https://issues.apache.org/jira/browse/CALCITE-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804165#comment-16804165 ] Julian Hyde commented on CALCITE-1515: -- [~Chunwei Lei], I did some fix-up (removing the redundant 'inputCount' argument to CURSOR, and allowing a table function to be called with zero inputs). Please take a look at https://github.com/julianhyde/calcite/commit/bd116f45fb9438293a55260717500391bf977197; if you +1 I will squash and commit. > Support TableFunctionScan in RelBuilder > --- > > Key: CALCITE-1515 > URL: https://issues.apache.org/jira/browse/CALCITE-1515 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.10.0 >Reporter: Anton Mushin >Assignee: Chunwei Lei >Priority: Minor > Labels: pull-request-available > Time Spent: 4.5h > Remaining Estimate: 0h > > Calcite has a TableFunctionScan, which is a RelNode, takes a function, 0 or > more RelNode inputs, and 0 or more other arguments. > RelBuilder does not support TableFunctionScan yet. > add In RelBuilder TableFunctionScanFactory which will create > LogicalTableFunctionScan like as for LogicalTableScan and factory for others > RelNodes [ > [example|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/tools/RelBuilder.java#L125] > ] > also adding a functionScan method, analogous to the scan method but for table > functions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-1581) UDTF like in hive
[
https://issues.apache.org/jira/browse/CALCITE-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804052#comment-16804052
]
pengzhiwei commented on CALCITE-1581:
-
Hi [~julianhyde], I have make a pr
[https://github.com/apache/calcite/pull/1138] for this issue.
In this pr,I mainly do the follow things:
* Rewrite the "parser.jj" file to support "udtf() as (f0,f1)" in the sql
parser stage with minimum modification.
* add allowSelectTableFunction() in SqlConformance and also add a kind of Hive
SqlConformance.
* In SqlValidator stage, I rewrite the "udtf() as (f0,f1)" to the
lateral-table sql-node. With this rewritting,this is nothing need to do in the
SqlToRelConverter.
> UDTF like in hive
> -
>
> Key: CALCITE-1581
> URL: https://issues.apache.org/jira/browse/CALCITE-1581
> Project: Calcite
> Issue Type: New Feature
>Reporter: Xiaoyong Deng
>Assignee: Sihua Zhou
>Priority: Major
> Labels: pull-request-available, udtf
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Support one row in and multi-column/multi-row out(one-to-many mapping), just
> like udtf in hive.
> The query would like this:
> {code}
> select
> func(c0, c1) as (f0, f1, f2)
> from table_name;
> {code}
> c0 and c1 are 'table_name' columns. f0, f1 and f2 are new generated columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-1581) UDTF like in hive
[
https://issues.apache.org/jira/browse/CALCITE-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-1581:
Labels: pull-request-available udtf (was: udtf)
> UDTF like in hive
> -
>
> Key: CALCITE-1581
> URL: https://issues.apache.org/jira/browse/CALCITE-1581
> Project: Calcite
> Issue Type: New Feature
>Reporter: Xiaoyong Deng
>Assignee: Sihua Zhou
>Priority: Major
> Labels: pull-request-available, udtf
>
> Support one row in and multi-column/multi-row out(one-to-many mapping), just
> like udtf in hive.
> The query would like this:
> {code}
> select
> func(c0, c1) as (f0, f1, f2)
> from table_name;
> {code}
> c0 and c1 are 'table_name' columns. f0, f1 and f2 are new generated columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2865) FilterProjectTransposeRule generates wrong traitSet when copyFilter/Project is true
[
https://issues.apache.org/jira/browse/CALCITE-2865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804041#comment-16804041
]
Haisheng Yuan commented on CALCITE-2865:
I am curious why does it provide copyFilter and copyProject option.
> FilterProjectTransposeRule generates wrong traitSet when copyFilter/Project
> is true
> ---
>
> Key: CALCITE-2865
> URL: https://issues.apache.org/jira/browse/CALCITE-2865
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Ruben Quesada Lopez
>Assignee: Ruben Quesada Lopez
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Problem can be reproduced with the following steps:
> As suggested by [~zabetak], a possibility to reproduce this issue is with the
> following test in RelOptRulesTest:
> {code:java}
> /** Test case for
>* href="https://issues.apache.org/jira/browse/CALCITE-2865;>[CALCITE-2865]
>* FilterProjectTransposeRule generates wrong traitSet when
> copyFilter/Project is true. */
> @Test public void testFilterProjectTransposeRule() {
> List rules = Arrays.asList(
> FilterProjectTransposeRule.INSTANCE, // default values:
> copyFilter=true, copyProject=true
> new FilterProjectTransposeRule(Filter.class, Project.class,
> /*copyFilter*/ false, /*copyProject*/ false, RelFactories.LOGICAL_BUILDER));
> List results = new ArrayList<>(2);
> for (RelOptRule rule : rules) {
> RelBuilder b = RelBuilder.create(RelBuilderTest.config().build());
> RelNode in = b
> .scan("EMP")
> .sort(-4) // salary desc
> .project(b.field(3)) // salary
> .filter(b.equals(b.field(0), b.literal(11500))) // salary =
> 11500
> .build();
> HepProgram program = new HepProgramBuilder()
> .addRuleInstance(rule)
> .build();
> HepPlanner hepPlanner = new HepPlanner(program);
> hepPlanner.setRoot(in);
> results.add(hepPlanner.findBestExp());
> }
> // compare LogicalFilter traitSet
> assertEquals(results.get(0).getInput(0).getTraitSet(),
> results.get(1).getInput(0).getTraitSet());
> }
> {code}
> Which fails with the following message:
> {code:java}
> java.lang.AssertionError:
> Expected :NONE.[0 DESC]
> Actual :NONE.[3 DESC]
> {code}
> If we run the test with a break point at the end of
> {{FilterProjectTransposeRule#onMatch}} method, we can see that, when the
> project and filter are transposed, the generated output (newFilterRel) is
> different in terms of traitSet, depending if the copyFilter / copyProject
> flags are used or not:
> - copyFilter and copyProject TRUE (default)
> {code:java}
> LogicalProject.NONE.[0 DESC]
> LogicalFilter.NONE.[0 DESC] // !!!
> LogicalSort.NONE.[3 DESC]
> {code}
> - copyFilter and copyProject FALSE
> {code:java}
> LogicalProject.NONE.[0 DESC]
> LogicalFilter.NONE.[3 DESC] // !!!
> LogicalSort.NONE.[3 DESC]
> {code}
> As I see it, the default behavior (copyFilter/Project true) seems wrong in
> terms of new filter's traitSet.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (CALCITE-2269) Enable Error Prone checking
[ https://issues.apache.org/jira/browse/CALCITE-2269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden resolved CALCITE-2269. --- Resolution: Fixed > Enable Error Prone checking > --- > > Key: CALCITE-2269 > URL: https://issues.apache.org/jira/browse/CALCITE-2269 > Project: Calcite > Issue Type: Improvement > Components: build >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Major > Labels: pull-request-available > Fix For: avatica-1.14.0 > > Time Spent: 50m > Remaining Estimate: 0h > > [http://errorprone.info/] provides the ability to do static checks like > Spotbugs (CALCITE-1006). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2269) Enable Error Prone checking
[ https://issues.apache.org/jira/browse/CALCITE-2269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803928#comment-16803928 ] Kevin Risden commented on CALCITE-2269: --- Merged in commit 1b68d1fae721f45405e580582abb8fede9a6c028 > Enable Error Prone checking > --- > > Key: CALCITE-2269 > URL: https://issues.apache.org/jira/browse/CALCITE-2269 > Project: Calcite > Issue Type: Improvement > Components: build >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Major > Labels: pull-request-available > Fix For: avatica-1.14.0 > > Time Spent: 50m > Remaining Estimate: 0h > > [http://errorprone.info/] provides the ability to do static checks like > Spotbugs (CALCITE-1006). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2866) Allow passing factory of SqlValidator / SqlToRelConverter / CatalogReader into FrameworkConfig
[ https://issues.apache.org/jira/browse/CALCITE-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated CALCITE-2866: -- Summary: Allow passing factory of SqlValidator / SqlToRelConverter / CatalogReader into FrameworkConfig (was: Allow passing factory of SqlValidator / SqlToRelConverter into FrameworkConfig) > Allow passing factory of SqlValidator / SqlToRelConverter / CatalogReader > into FrameworkConfig > --- > > Key: CALCITE-2866 > URL: https://issues.apache.org/jira/browse/CALCITE-2866 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Hongze Zhang >Assignee: Hongze Zhang >Priority: Major > Labels: usability > > It seems that the customization of SqlValidator / SqlToRelConverter / > CatalogReader in downstream projects is normal. > E.g. Following are some examples (not all) of extended SqlValidator / > SqlToRelConverter / CatalogReader from other projects that are using Calcite > as query planner : > # DrillValidator > > [https://github.com/apache/drill/blob/4627973bde9847a4eb2672c44941136c167326a1/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/SqlConverter.java#L249] > # FlinkCalciteSqlValidator > > [https://github.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/calcite/FlinkCalciteSqlValidator.scala#L31] > # DremioSqlToRelConverter > > [https://github.com/dremio/dremio-oss/blob/master/sabot/kernel/src/main/java/com/dremio/exec/planner/sql/DremioSqlToRelConverter.java#L40] > # (Dremio) SqlValidatorImpl > > [https://github.com/dremio/dremio-oss/blob/master/sabot/kernel/src/main/java/com/dremio/exec/planner/sql/SqlValidatorImpl.java#L32] > # DremioCatalogReader > > [https://github.com/dremio/dremio-oss/blob/master/sabot/kernel/src/main/java/com/dremio/exec/catalog/DremioCatalogReader.java#L71] > The class tools/Planner.java[1] is a good customization tool for Calcite > user. However currently there are no way to use Frameworks.getPlanner() > directly if the classes need to be customized[2][3]. > > [1] > [https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/tools/Planner.java] > [2] > [https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/prepare/PlannerImpl.java#L232] > [3] > [https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/prepare/PlannerImpl.java#L184] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2866) Allow passing factory of SqlValidator / SqlToRelConverter into FrameworkConfig
[ https://issues.apache.org/jira/browse/CALCITE-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated CALCITE-2866: -- Description: It seems that the customization of SqlValidator / SqlToRelConverter / CatalogReader in downstream projects is normal. E.g. Following are some examples (not all) of extended SqlValidator / SqlToRelConverter / CatalogReader from other projects that are using Calcite as query planner : # DrillValidator [https://github.com/apache/drill/blob/4627973bde9847a4eb2672c44941136c167326a1/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/SqlConverter.java#L249] # FlinkCalciteSqlValidator [https://github.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/calcite/FlinkCalciteSqlValidator.scala#L31] # DremioSqlToRelConverter [https://github.com/dremio/dremio-oss/blob/master/sabot/kernel/src/main/java/com/dremio/exec/planner/sql/DremioSqlToRelConverter.java#L40] # (Dremio) SqlValidatorImpl [https://github.com/dremio/dremio-oss/blob/master/sabot/kernel/src/main/java/com/dremio/exec/planner/sql/SqlValidatorImpl.java#L32] # DremioCatalogReader [https://github.com/dremio/dremio-oss/blob/master/sabot/kernel/src/main/java/com/dremio/exec/catalog/DremioCatalogReader.java#L71] The class tools/Planner.java[1] is a good customization tool for Calcite user. However currently there are no way to use Frameworks.getPlanner() directly if the classes need to be customized[2][3]. [1] [https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/tools/Planner.java] [2] [https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/prepare/PlannerImpl.java#L232] [3] [https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/prepare/PlannerImpl.java#L184] was: It seems that the customization of SqlValidator / SqlToRelConverter in downstream projects is normal. E.g. Following are some examples (not all) of extended SqlValidator or SqlToRelConverter from other projects that are using Calcite as query planner : # DrillValidator [https://github.com/apache/drill/blob/4627973bde9847a4eb2672c44941136c167326a1/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/SqlConverter.java#L249] # FlinkCalciteSqlValidator [https://github.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/calcite/FlinkCalciteSqlValidator.scala#L31] # DremioSqlToRelConverter [https://github.com/dremio/dremio-oss/blob/master/sabot/kernel/src/main/java/com/dremio/exec/planner/sql/DremioSqlToRelConverter.java#L40] # (Dremio) SqlValidatorImpl [https://github.com/dremio/dremio-oss/blob/master/sabot/kernel/src/main/java/com/dremio/exec/planner/sql/SqlValidatorImpl.java#L32] The class tools/Planner.java[1] is a good customization tool for Calcite user. However currently there are no way to use Frameworks.getPlanner() directly if the two class need to be customized[2][3]. [1] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/tools/Planner.java [2] [https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/prepare/PlannerImpl.java#L232] [3] [https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/prepare/PlannerImpl.java#L184] > Allow passing factory of SqlValidator / SqlToRelConverter into FrameworkConfig > -- > > Key: CALCITE-2866 > URL: https://issues.apache.org/jira/browse/CALCITE-2866 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Hongze Zhang >Assignee: Hongze Zhang >Priority: Major > Labels: usability > > It seems that the customization of SqlValidator / SqlToRelConverter / > CatalogReader in downstream projects is normal. > E.g. Following are some examples (not all) of extended SqlValidator / > SqlToRelConverter / CatalogReader from other projects that are using Calcite > as query planner : > # DrillValidator > > [https://github.com/apache/drill/blob/4627973bde9847a4eb2672c44941136c167326a1/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/SqlConverter.java#L249] > # FlinkCalciteSqlValidator > > [https://github.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/calcite/FlinkCalciteSqlValidator.scala#L31] > # DremioSqlToRelConverter > > [https://github.com/dremio/dremio-oss/blob/master/sabot/kernel/src/main/java/com/dremio/exec/planner/sql/DremioSqlToRelConverter.java#L40] > # (Dremio) SqlValidatorImpl > > [https://github.com/dremio/dremio-oss/blob/master/sabot/kernel/src/main/java/com/dremio/exec/planner/sql/SqlValidatorImpl.java#L32] > #
[jira] [Resolved] (CALCITE-2776) Wrong value when accessing struct types with one attribute
[
https://issues.apache.org/jira/browse/CALCITE-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stamatis Zampetakis resolved CALCITE-2776.
--
Resolution: Fixed
Fixed in
[5a305f166cf06419a0c28e496a9bf853ba540047|https://github.com/apache/calcite-avatica/commit/5a305f166cf06419a0c28e496a9bf853ba540047].
> Wrong value when accessing struct types with one attribute
> --
>
> Key: CALCITE-2776
> URL: https://issues.apache.org/jira/browse/CALCITE-2776
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.13.0
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Blocker
> Labels: pull-request-available
> Fix For: avatica-1.14.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The accessors for struct types with one attribute are not created correctly
> leading to wrong values when the internal representation for structured types
> is an Object[].
> For example consider the following query:
> {code:sql}
> select * from (values
> (1, ROW(1)),
> (2, ROW(2))) as v(id,struct);
> {code}
> Executing this query on calcite returns:
> {code:sql}
> +++
> | ID | STRUCT |
> +++
> | 1 | {[Ljava.lang.Object;@4ff4357f} |
> | 2 | {[Ljava.lang.Object;@49cb9cb5} |
> +++
> {code}
> instead of:
> {code:sql}
> +++
> | ID | STRUCT |
> +++
> | 1 | {1}|
> | 2 | {2}|
> +++
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2960) CalciteCatalogReader use a hard-coding config to get functions
[
https://issues.apache.org/jira/browse/CALCITE-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803624#comment-16803624
]
Danny Chan commented on CALCITE-2960:
-
BTW, i never saw any case that Calcite use CalciteCatalogReader to look up
operators, it only use it for table name resolving. For operators, Calcite
actually use SqlOperatorTable, please correct me if i'm wrong.
> CalciteCatalogReader use a hard-coding config to get functions
> --
>
> Key: CALCITE-2960
> URL: https://issues.apache.org/jira/browse/CALCITE-2960
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: Lai Zhou
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> CalciteCatalogReader use a hard-coding config to get functions
> {code:java}
> functions2.addAll(schema.getFunctions(name, true));
> {code}
> the right way is：
> {code:java}
> functions2.addAll(schema.getFunctions(name, config.caseSensitive()));
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2960) CalciteCatalogReader use a hard-coding config to get functions
[
https://issues.apache.org/jira/browse/CALCITE-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803618#comment-16803618
]
Danny Chan commented on CALCITE-2960:
-
I think it is part of issue 2928, and i have add link for it.
> CalciteCatalogReader use a hard-coding config to get functions
> --
>
> Key: CALCITE-2960
> URL: https://issues.apache.org/jira/browse/CALCITE-2960
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: Lai Zhou
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> CalciteCatalogReader use a hard-coding config to get functions
> {code:java}
> functions2.addAll(schema.getFunctions(name, true));
> {code}
> the right way is：
> {code:java}
> functions2.addAll(schema.getFunctions(name, config.caseSensitive()));
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2928) Make UDF lookup default to case insensitive
[
https://issues.apache.org/jira/browse/CALCITE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803610#comment-16803610
]
Danny Chan edited comment on CALCITE-2928 at 3/28/19 6:09 AM:
--
[~julianhyde] i have fire a
[PR#1137|https://github.com/apache/calcite/pull/1137]
Initially i wanna change both builtin operators and UDFs case sensitive
internal, and control the case-sensitiveness through config parser config[2].
The problems i encounter when doing this patch:
1.There is a SqlAbstractParserImpl which extended by all the SqlParser, so
there is no change to see the case-sensitivless when parsing the builtin
operators in SqlStdOperatorTable[1], and i have to hard code the case
sensitiveness to be false.
2. Because of 1 i have to make the
ReflectiveSqlOperatorTable#lookupOperatorOverloads case- insensitive for
SqlStdOperatorTable(use instance of to detect).
So with this patch, we can control both table/column/UDF name with
SqlParser.Config.caseSensitive(this flag can be configured through connection
property or FrameworkConfig).
But for builtins, they behave all case-insensitively, just like before.
BTW, this patch add the flag in SqlOperatorTable#lookupOperatorOverloads, so
many codes are touched, i kind of thought the SqlOperatorTable is also
plugable, if we create a case-insensitive ListSqlOperatorTable can also make
another choice for users. And this will make minor change.
[1]
[link1|https://github.com/apache/calcite/blob/3124a85b93ff2f1b79484c7bd4cc41835d4f1920/core/src/main/java/org/apache/calcite/sql/parser/SqlAbstractParserImpl.java#L395]
[2]
[link2|https://github.com/apache/calcite/blob/3124a85b93ff2f1b79484c7bd4cc41835d4f1920/core/src/main/java/org/apache/calcite/sql/parser/SqlParser.java#L221]
was (Author: danny0405):
[~julianhyde] i have fire a
[PR#1137|https://github.com/apache/calcite/pull/1137]
Initially i wanna change both builtin operators and UDFs case sensitive
internal, and control the case-sensitiveness through config parser config[2].
The problems i encounter when doing this patch:
1.There is a SqlAbstractParserImpl which extended by all the SqlParser, so
there is no change to see the case-sensitivless when parsing the builtin
operators in SqlStdOperatorTable[1], and i have to hard code the case
sensitiveness to be false.
2. Because of 1 i have to make the
ReflectiveSqlOperatorTable#lookupOperatorOverloads case- insensitive for
SqlStdOperatorTable(use instance of to detect).
So with this patch, we can control both table/column/UDF name with
SqlParser.Config.caseSensitive(this flag can be configured through connection
property or FrameworkConfig).
But for builtins, they behave all case-insensitively, just like before.
BTW, this patch add the flag in SqlOperatorTable#lookupOperatorOverloads, so
many codes are touched, i kind of thought the SqlOperatorTable is also
plugable, if we create a case-insensitive ListSqlOperatorTable can also make
another choice for users. And this will make minor change.
[1]
[link1|https://github.com/apache/calcite/blob/3124a85b93ff2f1b79484c7bd4cc41835d4f1920/core/src/main/java/org/apache/calcite/sql/parser/SqlAbstractParserImpl.java#L395]
[2][link2|https://github.com/apache/calcite/blob/3124a85b93ff2f1b79484c7bd4cc41835d4f1920/core/src/main/java/org/apache/calcite/sql/parser/SqlParser.java#L221]
> Make UDF lookup default to case insensitive
> ---
>
> Key: CALCITE-2928
> URL: https://issues.apache.org/jira/browse/CALCITE-2928
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.19.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Now for Calcite, we make default parser config unquotedCasing to
> Lex.ORACLE.unquotedCasing(to uppercase)[1], and caseSensitive to
> Lex.ORACLE.caseSensitive(case sensitive true).
> So if we have a UDAF named my_func and query with sql like:
> {code:java}
> select f0, my_func(f1) from table1 group by f0;
> {code}
> We would got a unparsed sql:
> {code:java}
> SELECT F0, MY_FUNC(F1) FROM TABLE1 GROUP BY F0;
> {code}
> For CalciteCatalogReader we hard code the function look up to case sensitive
> true[2],
> For ListSqlOperatorTable we make the operator name lookup case sensitive
> true[3].
> For ReflectiveSqlOperatorTable, we make built-in operators
> case-insensitively[4].
> For most of the cases, we use ListSqlOperatorTable to register our UDFs[5]
> chained with SqlStdOperatorTable(which composite a ChainedSqlOperatorTable),
> which finally passed to CalciteCatalogReader for validation.
> So there are some questions i have:
> 1. Why we make built-in operators look
[jira] [Comment Edited] (CALCITE-2928) Make UDF lookup default to case insensitive
[
https://issues.apache.org/jira/browse/CALCITE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803610#comment-16803610
]
Danny Chan edited comment on CALCITE-2928 at 3/28/19 6:08 AM:
--
[~julianhyde] i have fire a
[PR#1137|https://github.com/apache/calcite/pull/1137]
Initially i wanna change both builtin operators and UDFs case sensitive
internal, and control the case-sensitiveness through config parser config[2].
The problems i encounter when doing this patch:
1.There is a SqlAbstractParserImpl which extended by all the SqlParser, so
there is no change to see the case-sensitivless when parsing the builtin
operators in SqlStdOperatorTable[1], and i have to hard code the case
sensitiveness to be false.
2. Because of 1 i have to make the
ReflectiveSqlOperatorTable#lookupOperatorOverloads case- insensitive for
SqlStdOperatorTable(use instance of to detect).
So with this patch, we can control both table/column/UDF name with
SqlParser.Config.caseSensitive(this flag can be configured through connection
property or FrameworkConfig).
But for builtins, they behave all case-insensitively, just like before.
BTW, this patch add the flag in SqlOperatorTable#lookupOperatorOverloads, so
many codes are touched, i kind of thought the SqlOperatorTable is also
plugable, if we create a case-insensitive ListSqlOperatorTable can also make
another choice for users. And this will make minor change.
[1]
[link1|https://github.com/apache/calcite/blob/3124a85b93ff2f1b79484c7bd4cc41835d4f1920/core/src/main/java/org/apache/calcite/sql/parser/SqlAbstractParserImpl.java#L395]
[2][link2|https://github.com/apache/calcite/blob/3124a85b93ff2f1b79484c7bd4cc41835d4f1920/core/src/main/java/org/apache/calcite/sql/parser/SqlParser.java#L221]
was (Author: danny0405):
[~julianhyde] i have fire a
[PR#1137|https://github.com/apache/calcite/pull/1137]
Initially i wanna change both builtin operators and UDFs case sensitive
internal, and control the case-sensitiveness through config parser config[2].
The problems i encounter when doing this patch:
1.There is a SqlAbstractParserImpl which extended by all the SqlParser, so
there is no change to see the case-sensitivless when parsing the builtin
operators in SqlStdOperatorTable[1], and i have to hard code the case
sensitiveness to be false.
2. Because of 1 i have to make the
ReflectiveSqlOperatorTable#lookupOperatorOverloads case- insensitive for
SqlStdOperatorTable(use instance of to detect).
So with this patch, we can control both table/column/UDF name with
SqlParser.Config.caseSensitive(this flag can be configured through connection
property or FrameworkConfig).
But for builtins, they behave all case-insensitively, just like before.
BTW, this patch add the flag in SqlOperatorTable#lookupOperatorOverloads, so
many codes are touched, i kind of thought the SqlOperatorTable is also
plugable, if we create a case-insensitive ListSqlOperatorTable can also make
another choice for users. And this will make minor change.
[1]https://github.com/apache/calcite/blob/3124a85b93ff2f1b79484c7bd4cc41835d4f1920/core/src/main/java/org/apache/calcite/sql/parser/SqlAbstractParserImpl.java#L395
[2]https://github.com/apache/calcite/blob/3124a85b93ff2f1b79484c7bd4cc41835d4f1920/core/src/main/java/org/apache/calcite/sql/parser/SqlParser.java#L221
> Make UDF lookup default to case insensitive
> ---
>
> Key: CALCITE-2928
> URL: https://issues.apache.org/jira/browse/CALCITE-2928
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.19.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Now for Calcite, we make default parser config unquotedCasing to
> Lex.ORACLE.unquotedCasing(to uppercase)[1], and caseSensitive to
> Lex.ORACLE.caseSensitive(case sensitive true).
> So if we have a UDAF named my_func and query with sql like:
> {code:java}
> select f0, my_func(f1) from table1 group by f0;
> {code}
> We would got a unparsed sql:
> {code:java}
> SELECT F0, MY_FUNC(F1) FROM TABLE1 GROUP BY F0;
> {code}
> For CalciteCatalogReader we hard code the function look up to case sensitive
> true[2],
> For ListSqlOperatorTable we make the operator name lookup case sensitive
> true[3].
> For ReflectiveSqlOperatorTable, we make built-in operators
> case-insensitively[4].
> For most of the cases, we use ListSqlOperatorTable to register our UDFs[5]
> chained with SqlStdOperatorTable(which composite a ChainedSqlOperatorTable),
> which finally passed to CalciteCatalogReader for validation.
> So there are some questions i have:
> 1. Why we make built-in operators look up
[jira] [Commented] (CALCITE-2928) Make UDF lookup default to case insensitive
[
https://issues.apache.org/jira/browse/CALCITE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803610#comment-16803610
]
Danny Chan commented on CALCITE-2928:
-
[~julianhyde] i have fire a
[PR#1137|https://github.com/apache/calcite/pull/1137]
Initially i wanna change both builtin operators and UDFs case sensitive
internal, and control the case-sensitiveness through config parser config[2].
The problems i encounter when doing this patch:
1.There is a SqlAbstractParserImpl which extended by all the SqlParser, so
there is no change to see the case-sensitivless when parsing the builtin
operators in SqlStdOperatorTable[1], and i have to hard code the case
sensitiveness to be false.
2. Because of 1 i have to make the
ReflectiveSqlOperatorTable#lookupOperatorOverloads case- insensitive for
SqlStdOperatorTable(use instance of to detect).
So with this patch, we can control both table/column/UDF name with
SqlParser.Config.caseSensitive(this flag can be configured through connection
property or FrameworkConfig).
But for builtins, they behave all case-insensitively, just like before.
BTW, this patch add the flag in SqlOperatorTable#lookupOperatorOverloads, so
many codes are touched, i kind of thought the SqlOperatorTable is also
plugable, if we create a case-insensitive ListSqlOperatorTable can also make
another choice for users. And this will make minor change.
[1]https://github.com/apache/calcite/blob/3124a85b93ff2f1b79484c7bd4cc41835d4f1920/core/src/main/java/org/apache/calcite/sql/parser/SqlAbstractParserImpl.java#L395
[2]https://github.com/apache/calcite/blob/3124a85b93ff2f1b79484c7bd4cc41835d4f1920/core/src/main/java/org/apache/calcite/sql/parser/SqlParser.java#L221
> Make UDF lookup default to case insensitive
> ---
>
> Key: CALCITE-2928
> URL: https://issues.apache.org/jira/browse/CALCITE-2928
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.19.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Now for Calcite, we make default parser config unquotedCasing to
> Lex.ORACLE.unquotedCasing(to uppercase)[1], and caseSensitive to
> Lex.ORACLE.caseSensitive(case sensitive true).
> So if we have a UDAF named my_func and query with sql like:
> {code:java}
> select f0, my_func(f1) from table1 group by f0;
> {code}
> We would got a unparsed sql:
> {code:java}
> SELECT F0, MY_FUNC(F1) FROM TABLE1 GROUP BY F0;
> {code}
> For CalciteCatalogReader we hard code the function look up to case sensitive
> true[2],
> For ListSqlOperatorTable we make the operator name lookup case sensitive
> true[3].
> For ReflectiveSqlOperatorTable, we make built-in operators
> case-insensitively[4].
> For most of the cases, we use ListSqlOperatorTable to register our UDFs[5]
> chained with SqlStdOperatorTable(which composite a ChainedSqlOperatorTable),
> which finally passed to CalciteCatalogReader for validation.
> So there are some questions i have:
> 1. Why we make built-in operators look up case-insensitively while
> ListSqlOperatorTable(for UDFs) case-sensitively, with default unquotedCasing
> of TO_UPPERCASE.
> 2. What is the usage of CalciteCatalogReader#lookupOperatorOverloads i only
> saw it used in a unit test LookupOperatorOverloadsTest.
> It seems that make UDF look up case-sensitively does not make any sense,
> users will never distinguish their function with just word cases. And i
> checked also MYSQL, ORACLE, POSTGRES, their UDFs are all registered
> case-insensitively.
> [1]
> https://github.com/apache/calcite/blob/ffca956be03a99cd11e440d652b09674aaa727e6/core/src/main/java/org/apache/calcite/sql/parser/SqlParser.java#L231
> [2]
> https://github.com/apache/calcite/blob/ffca956be03a99cd11e440d652b09674aaa727e6/core/src/main/java/org/apache/calcite/prepare/CalciteCatalogReader.java#L166
> [3]
> https://github.com/apache/calcite/blob/ffca956be03a99cd11e440d652b09674aaa727e6/core/src/main/java/org/apache/calcite/sql/util/ListSqlOperatorTable.java#L63
> [4]
> https://github.com/apache/calcite/blob/ffca956be03a99cd11e440d652b09674aaa727e6/core/src/main/java/org/apache/calcite/sql/util/ReflectiveSqlOperatorTable.java#L103
> [5]
> https://github.com/apache/calcite/blob/ffca956be03a99cd11e440d652b09674aaa727e6/core/src/test/java/org/apache/calcite/test/MockSqlOperatorTable.java#L46
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2928) Make UDF lookup default to case insensitive
[
https://issues.apache.org/jira/browse/CALCITE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-2928:
Labels: pull-request-available (was: )
> Make UDF lookup default to case insensitive
> ---
>
> Key: CALCITE-2928
> URL: https://issues.apache.org/jira/browse/CALCITE-2928
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.19.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
>
> Now for Calcite, we make default parser config unquotedCasing to
> Lex.ORACLE.unquotedCasing(to uppercase)[1], and caseSensitive to
> Lex.ORACLE.caseSensitive(case sensitive true).
> So if we have a UDAF named my_func and query with sql like:
> {code:java}
> select f0, my_func(f1) from table1 group by f0;
> {code}
> We would got a unparsed sql:
> {code:java}
> SELECT F0, MY_FUNC(F1) FROM TABLE1 GROUP BY F0;
> {code}
> For CalciteCatalogReader we hard code the function look up to case sensitive
> true[2],
> For ListSqlOperatorTable we make the operator name lookup case sensitive
> true[3].
> For ReflectiveSqlOperatorTable, we make built-in operators
> case-insensitively[4].
> For most of the cases, we use ListSqlOperatorTable to register our UDFs[5]
> chained with SqlStdOperatorTable(which composite a ChainedSqlOperatorTable),
> which finally passed to CalciteCatalogReader for validation.
> So there are some questions i have:
> 1. Why we make built-in operators look up case-insensitively while
> ListSqlOperatorTable(for UDFs) case-sensitively, with default unquotedCasing
> of TO_UPPERCASE.
> 2. What is the usage of CalciteCatalogReader#lookupOperatorOverloads i only
> saw it used in a unit test LookupOperatorOverloadsTest.
> It seems that make UDF look up case-sensitively does not make any sense,
> users will never distinguish their function with just word cases. And i
> checked also MYSQL, ORACLE, POSTGRES, their UDFs are all registered
> case-insensitively.
> [1]
> https://github.com/apache/calcite/blob/ffca956be03a99cd11e440d652b09674aaa727e6/core/src/main/java/org/apache/calcite/sql/parser/SqlParser.java#L231
> [2]
> https://github.com/apache/calcite/blob/ffca956be03a99cd11e440d652b09674aaa727e6/core/src/main/java/org/apache/calcite/prepare/CalciteCatalogReader.java#L166
> [3]
> https://github.com/apache/calcite/blob/ffca956be03a99cd11e440d652b09674aaa727e6/core/src/main/java/org/apache/calcite/sql/util/ListSqlOperatorTable.java#L63
> [4]
> https://github.com/apache/calcite/blob/ffca956be03a99cd11e440d652b09674aaa727e6/core/src/main/java/org/apache/calcite/sql/util/ReflectiveSqlOperatorTable.java#L103
> [5]
> https://github.com/apache/calcite/blob/ffca956be03a99cd11e440d652b09674aaa727e6/core/src/test/java/org/apache/calcite/test/MockSqlOperatorTable.java#L46
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
