[jira] [Created] (CALCITE-2317) getFunctions() showing empty
Subbarao created CALCITE-2317: - Summary: getFunctions() showing empty Key: CALCITE-2317 URL: https://issues.apache.org/jira/browse/CALCITE-2317 Project: Calcite Issue Type: Bug Reporter: Subbarao Assignee: Julian Hyde how to get all function names and procedure names in oracle database using calcite connection and schema. schema.getFunctionNames() return result is:[ ] Then i am try to retrieving DatabaseMetadata.getFunction().Here also it will be showing empty. Iam using calciResultSet ,AvaticaResultSet and ResultSet for getting functions and procedures but will be showing empty. Is it possible get all function names and procedure names from any database. But normal jdbc connection using DatabseMetaData properties it retrive alll function names and procedure names.Why it will not retrive in calcite -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2315) Inner-join query failed : Java.lang.AssertionError
[ https://issues.apache.org/jira/browse/CALCITE-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiaorui updated CALCITE-2315: - Affects Version/s: 1.15.0 Component/s: elasticsearch-adapter > Inner-join query failed : Java.lang.AssertionError > > > Key: CALCITE-2315 > URL: https://issues.apache.org/jira/browse/CALCITE-2315 > Project: Calcite > Issue Type: Bug > Components: elasticsearch-adapter >Affects Versions: 1.15.0 >Reporter: xiaorui >Assignee: Julian Hyde >Priority: Major > Attachments: logs > > > > "select table1.\"int1\",table1.\"str1\",table2.\"int1\",table2.\"str1\" from > table1 inner join table2 on table1.\"id\" = table2.\"id\" where table1.\"id\" > = 1" > AssertionError would be thrown out When I try to run the above SQL ,The stack > is as following (Detailed log see attachment). > java.lang.AssertionError > at > org.apache.calcite.adapter.elasticsearch.ElasticsearchProject.(ElasticsearchProject.java:44) > at > org.apache.calcite.adapter.elasticsearch.ElasticsearchProject.copy(ElasticsearchProject.java:49) > at > org.apache.calcite.rel.rules.FilterProjectTransposeRule.onMatch(FilterProjectTransposeRule.java:131) > at > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:650) > at org.apache.calcite.tools.Programs$5.run(Programs.java:326) > at > org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:387) > at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:188) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:319) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:230) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:781) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610) > at > org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:221) > at > org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603) > at > org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638) > at > org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149) > at > org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:209) > at > org.apache.calcite.test.ElasticsearchAdapterTest.testElasticSearch(ElasticsearchAdapterTest.java:35) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) > at org.junit.runner.JUnitCore.run(JUnitCore.java:137) > at > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) > at > com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) > at > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) > at > com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2316) ES adapter converts query literals to lowercase
[
https://issues.apache.org/jira/browse/CALCITE-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Sereda updated CALCITE-2316:
---
Summary: ES adapter converts query literals to lowercase (was: ES adapter
converts query to lowercase)
> ES adapter converts query literals to lowercase
> ---
>
> Key: CALCITE-2316
> URL: https://issues.apache.org/jira/browse/CALCITE-2316
> Project: Calcite
> Issue Type: Bug
> Components: elasticsearch-adapter
>Reporter: Andrei Sereda
>Assignee: Julian Hyde
>Priority: Major
>
> Example of usage
> SQL: select * from "elastic" where _MAP['Foo'] = 'BAR' (note upper-case)
> ES Query: \{ "term": { "foo" : "bar" }} (note lower-case)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2316) ES adapter converts query to lowercase
[
https://issues.apache.org/jira/browse/CALCITE-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Sereda updated CALCITE-2316:
---
Description:
Example of usage
SQL: select * from "elastic" where _MAP['Foo'] = 'BAR' (note upper-case)
ES Query: \{ "term": { "foo" : "bar" }} (note lower-case)
was:
Example of usage
{{SQL: select * from "elastic" where _MAP['Foo'] = 'BAR' (note upper-case) }}
{{ES Query: \{ "term": { "foo" : "bar" }} (note lower-case) }}
> ES adapter converts query to lowercase
> --
>
> Key: CALCITE-2316
> URL: https://issues.apache.org/jira/browse/CALCITE-2316
> Project: Calcite
> Issue Type: Bug
> Components: elasticsearch-adapter
>Reporter: Andrei Sereda
>Assignee: Julian Hyde
>Priority: Major
>
> Example of usage
> SQL: select * from "elastic" where _MAP['Foo'] = 'BAR' (note upper-case)
> ES Query: \{ "term": { "foo" : "bar" }} (note lower-case)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2316) ES adapter converts query to lowercase
Andrei Sereda created CALCITE-2316:
--
Summary: ES adapter converts query to lowercase
Key: CALCITE-2316
URL: https://issues.apache.org/jira/browse/CALCITE-2316
Project: Calcite
Issue Type: Bug
Components: elasticsearch-adapter
Reporter: Andrei Sereda
Assignee: Julian Hyde
Example of usage
{{SQL: select * from "elastic" where _MAP['Foo'] = 'BAR' (note upper-case) }}
{{ES Query: \{ "term": { "foo" : "bar" }} (note lower-case) }}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2247) Add rule to push in condition condition into a related disjunctive expression
[
https://issues.apache.org/jira/browse/CALCITE-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479871#comment-16479871
]
Julian Hyde commented on CALCITE-2247:
--
I've reviewed again, and I still have one problem. It is not valid to simplify
{{x <> 1 OR x = 1}} to TRUE, even with unknownAsFalse.
If x is NULL, that expression evaluates to UNKNOWN. If unknownAsFalse, then
that UNKNOWN becomes FALSE.
I rebased your change on top of my dev branch for CALCITE-2314 (see
https://github.com/julianhyde/calcite/tree/2314-verify-rex), and that issue
shows up as the sole failing test.
> Add rule to push in condition condition into a related disjunctive expression
> -
>
> Key: CALCITE-2247
> URL: https://issues.apache.org/jira/browse/CALCITE-2247
> Project: Calcite
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
>
> Simplify expressions like: {code}a = 1 AND (a = 1 OR a = 2){code} to {code}a
> = 1{code}
> Conditions to apply will be:
> * in an AND condition there exists a comparison(c) and an OR (o)
> * o and c only reference 1 variable
> See HIVE-19097 for more info.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2158) SubQuery with EXISTS clause creates redundant aggregate call
[
https://issues.apache.org/jira/browse/CALCITE-2158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479512#comment-16479512
]
Julian Hyde commented on CALCITE-2158:
--
Thanks for all the detective work. I believe the modern approach – namely,
{{SqlToRelConverter.Config.isExpand()}} is false, and use
{{SubQueryRemoveRule}} – is better and has fewer bugs. So I recommend using
that, and would prioritize fixing the bugs in that code path.
> SubQuery with EXISTS clause creates redundant aggregate call
>
>
> Key: CALCITE-2158
> URL: https://issues.apache.org/jira/browse/CALCITE-2158
> Project: Calcite
> Issue Type: Bug
>Reporter: Volodymyr Vysotskyi
>Assignee: Julian Hyde
>Priority: Major
>
> When {{SqlToRelConverter.Config.isExpand()}} returns true, subqueries are
> expanded in {{SqlToRelConverter}}.
> Then for the queries, like this:
> {code:sql}
> SELECT cs1.sal
> FROM emp cs1
> WHERE EXISTS
> (SELECT *
> FROM emp cs2
> WHERE cs1.sal = cs2.sal
>AND cs1.deptno <> cs2.deptno)
> {code}
> Calcite returns logical plan with excessive aggregate calls:
> {noformat}
> LogicalProject(SAL=[$5])
> LogicalFilter(condition=[IS NOT NULL($9)])
> LogicalCorrelate(correlation=[$cor0], joinType=[left],
> requiredColumns=[{5, 7}])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalAggregate(group=[{}], agg#0=[MIN($0)])
> LogicalProject($f0=[true])
> LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3],
> HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
> LogicalFilter(condition=[AND(=($cor0.SAL, $5), <>($cor0.DEPTNO,
> $7))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {noformat}
> But when {{SqlToRelConverter.Config.isExpand()}} returns false and
> SubQueryRemoveRule rules are applied to the logical plan with RexSubQuery,
> the resulting logical plan is correct and does not contain excessive
> aggregate calls:
> {noformat}
> LogicalProject(SAL=[$5])
> LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
> SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
> LogicalCorrelate(correlation=[$cor0], joinType=[inner],
> requiredColumns=[{5, 7}])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalAggregate(group=[{0}])
> LogicalProject(i=[true])
> LogicalFilter(condition=[AND(=($cor0.SAL, $5), <>($cor0.DEPTNO,
> $7))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {noformat}
> These cases may be observed using this unit test:
> {code:java}
> @Test public void testExistsExpand() {
> final HepProgram preProgram = HepProgram.builder()
> .addRuleInstance(SubQueryRemoveRule.FILTER)
> .addRuleInstance(SubQueryRemoveRule.PROJECT)
> .addRuleInstance(SubQueryRemoveRule.JOIN)
> .build();
> final HepProgram program = HepProgram.builder()
> .build();
> final String sql = "SELECT cs1.sal\n"
> + "FROM emp cs1 \n"
> + "WHEREEXISTS\n"
> + "(SELECT *\n"
> + "FROM emp cs2\n"
> + "WHERE cs1.sal = cs2.sal\n"
> + "ANDcs1.deptno <> cs2.deptno)";
> sql(sql)
> .withDecorrelation(false)
> .withTrim(false)
> .expand(true) // change to false
> .withPre(preProgram)
> .with(program)
> .checkUnchanged();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2158) SubQuery with EXISTS clause creates redundant aggregate call
[
https://issues.apache.org/jira/browse/CALCITE-2158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479476#comment-16479476
]
Volodymyr Vysotskyi commented on CALCITE-2158:
--
Realized that this aggregate call may be used to check that aggregate call
returns the single value, so it is not needed to remove it from the aggregate.
But the real problem appears in the case when the query has several correlated
subqueries. This test:
{code:java}
@Test public void testRelDecorrelatorWithComplexFilters() {
final HepProgram program = HepProgram.builder()
.build();
final String sql = "SELECT cs1.sal\n"
+ "FROM emp cs1,\n"
+ " emp cs3\n"
+ "WHEREcs1.ename = cs3.ename\n"
+ "AND EXISTS\n"
+ "(SELECT *\n"
+ "FROM emp cs2\n"
+ "WHERE cs1.sal = cs2.sal\n"
+ "ANDcs1.deptno <> cs2.deptno)"
+ "AND EXISTS"
+ "(SELECT *\n"
+ "FROM emp cr1\n"
+ "WHERE cs1.sal = cr1.sal)";
sql(sql)
.withDecorrelation(true)
.expand(true)
.with(program)
.checkUnchanged();
}
{code}
Returns plan with join with true in condition
{noformat}
LogicalProject(SAL=[$5])
LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], EMPNO0=[$9], ENAME0=[$10],
JOB0=[$11], MGR0=[$12], HIREDATE0=[$13], SAL0=[$14], COMM0=[$15],
DEPTNO0=[$16], SLACKER0=[$17], $f0=[$18], $f019=[$20])
LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], EMPNO0=[$9], ENAME0=[$10],
JOB0=[$11], MGR0=[$12], HIREDATE0=[$13], SAL0=[$14], COMM0=[$15],
DEPTNO0=[$16], SLACKER0=[$17], $f0=[$18], SAL9=[CAST($19):INTEGER],
$f1=[CAST($20):BOOLEAN])
LogicalJoin(condition=[=($5, $19)], joinType=[inner])
LogicalFilter(condition=[AND(=($1, $10), IS NOT NULL($18))])
LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3],
HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], EMPNO0=[$9],
ENAME0=[$10], JOB0=[$11], MGR0=[$12], HIREDATE0=[$13], SAL0=[$14], COMM0=[$15],
DEPTNO0=[$16], SLACKER0=[$17], $f0=[$20])
LogicalJoin(condition=[AND(=($5, $18), =($7, $19))],
joinType=[left])
LogicalJoin(condition=[true], joinType=[inner])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalAggregate(group=[{0, 1}], agg#0=[MIN($2)])
LogicalProject(SAL0=[$1], DEPTNO0=[$2], $f0=[$0])
LogicalProject($f0=[true], SAL0=[$9], DEPTNO0=[$10])
LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3],
HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], SAL0=[$9],
DEPTNO0=[$10])
LogicalJoin(condition=[AND(=($9, $5), <>($10, $7))],
joinType=[inner])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalAggregate(group=[{0, 1}])
LogicalProject(SAL=[$5], DEPTNO=[$7])
LogicalJoin(condition=[true], joinType=[inner])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalAggregate(group=[{0}], agg#0=[MIN($1)])
LogicalProject(SAL9=[$1], $f0=[$0])
LogicalProject($f0=[true], SAL9=[$9])
LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3],
HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], SAL9=[$5])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{noformat}
It happens because {{HepPlanner}} in {{RelDecorrelator}} partially optimizes
input plan:
{noformat}
LogicalProject(SAL=[$5]): rowcount = 1.0, cumulative cost = 4049.7501,
id = 67
LogicalFilter(condition=[AND(=($1, $10), IS NOT NULL($18), IS NOT
NULL($19))]): rowcount = 1.0, cumulative cost = 4048.7501, id = 65
LogicalCorrelate(correlation=[$cor2], joinType=[left],
requiredColumns=[{5}]): rowcount = 1.0, cumulative cost = 4047.7501, id
= 63
LogicalCorrelate(correlation=[$cor0], joinType=[left],
requiredColumns=[{5, 7}]): rowcount = 1.0, cumulative cost =
4002.90005, id = 52
LogicalJoin(condition=[true], joinType=[inner]): rowcount = 196.0,
cumulative cost = 224.0, id = 41
LogicalTableScan(table=[[CATALOG, SALES, EMP]]): rowcount = 14.0,
cumulative cost = 14.0, id = 22
LogicalTableScan(table=[[CATALOG, SALES, EMP]]): rowcount = 14.0,
cumulative cost = 14.0, id = 23
LogicalAggregate(group=[{}], agg#0=[MIN($0)]): rowcount = 1.0,
cumulative cost = 18.2750002, id = 50
LogicalProject($f0=[true]): rowcount = 1.05, cumulative cost =
17.152, id =
[jira] [Commented] (CALCITE-2298) Correlated SubQuery in Having generates error plan when correlated fields does not exist
[
https://issues.apache.org/jira/browse/CALCITE-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479421#comment-16479421
]
Julian Hyde commented on CALCITE-2298:
--
I've reached deadlock on this. I just can't believe that this much new code is
needed. But I don't have time to go through your code and try to simplify it.
> Correlated SubQuery in Having generates error plan when correlated fields
> does not exist
>
>
> Key: CALCITE-2298
> URL: https://issues.apache.org/jira/browse/CALCITE-2298
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.15.0
>Reporter: godfrey he
>Assignee: Julian Hyde
>Priority: Major
>
> {code}
> @Test public void testDecorrelateWithUnresolvedField() throws Exception {
> final String sql = "select ename\n"
>+ "from sales.emp\n"
>+ "group by ename, deptno\n"
>+ "having max(sal) <= \n"
>+ " (select max(sal) from sales.emp_b where emp.job =
> emp_b.job group by ename)";
> checkSubQuery(sql).withLateDecorrelation(true).check();
> }
> {code}
> for now, we will get the following plan:
> {code}
> LogicalProject(ENAME=[$0])
> LogicalFilter(condition=[<=($2, $SCALAR_QUERY({
> LogicalProject(EXPR$0=[$1])
> LogicalAggregate(group=[{0}], EXPR$0=[MAX($1)])
> LogicalProject(ENAME=[$1], SAL=[$5])
> LogicalFilter(condition=[=($cor0.JOB, $2)])
> LogicalTableScan(table=[[CATALOG, SALES, EMP_B]])
> }))])
> LogicalAggregate(group=[{0, 1}], agg#0=[MAX($2)])
> LogicalProject(ENAME=[$1], DEPTNO=[$7], SAL=[$5])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> However emp.job is not the grouping fields or the aggCall result.
> the expected result is throwing an Exception, like:
> {code}
> org.apache.calcite.runtime.CalciteContextException: From line 5, column 47 to
> line 5, column 50: Column 'JOB' not found in 'LogicalAggregate(group=[{0,
> 1}], agg#0=[MAX($2)])'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2312) Support Partition By in sql select statement
[
https://issues.apache.org/jira/browse/CALCITE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479409#comment-16479409
]
Julian Hyde commented on CALCITE-2312:
--
That sounds like a plan. You probably want to represent other partitioning
schemes (e.g. round robin, singleton) as well as partitioned by keys. I don't
think we'd add this to Calcite's DDL parser but if you can share what you come
up with, it might be interesting to other people.
> Support Partition By in sql select statement
>
>
> Key: CALCITE-2312
> URL: https://issues.apache.org/jira/browse/CALCITE-2312
> Project: Calcite
> Issue Type: New Feature
> Components: core
>Reporter: Fei Xu
>Assignee: Julian Hyde
>Priority: Major
>
> I noticed that calcite already have an Exchange RelNode represents
> distribution on RelNode's input, but no sql clause support.
>
> We are use calcite building SQL layer for out streaming platform. and in
> streaming computation, data shuffle is a very important function, not only
> for our engine, but also for our users.
> For example, In stream-join-table, the engine will load the table data to a
> cache at runtime, and stream join table is actually stream look up table.
> If the stream data could partition by hash or range, before look up table. It
> will be cache friendly cause particular stream data look up particular table
> data.
>
> So I consider do some extensions on sql clause to support Exchange, e.g.
> {code:java}
> // shuffle data using hash_distribution
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY o.productId
> // shuffle data to a singleton node
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY SINGLETON
> // shuffle data to all nodes
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY BROADCAST
> // shuffle data to random node
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY RANDOM
> // shuffle data using round-robin policy
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY ROUND_ROBIN
> // shuffle data using range policy
> // Current I'm not sure about the appropriate clause to represents range
> // shuffle, so it is just an demo.
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY RANGE o.productId, 0, 4096
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-500) Ensure EnumerableJoin hashes the smallest input
[
https://issues.apache.org/jira/browse/CALCITE-500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479139#comment-16479139
]
Atri Sharma commented on CALCITE-500:
-
Fixed now, please review and let me know
> Ensure EnumerableJoin hashes the smallest input
> ---
>
> Key: CALCITE-500
> URL: https://issues.apache.org/jira/browse/CALCITE-500
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.0.0-incubating
>Reporter: Vladimir Sitnikov
>Assignee: Atri Sharma
>Priority: Major
> Labels: newbie
> Attachments: lookup.png
>
>
> {{EnumerableJoin}} tries to put the smallest input the first, however when it
> comes to execution, Calcite creates lookup for _second_ input of join.
> It would be nice to ensure the lookup is created on the smallest input.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2315) Inner-join query failed : Java.lang.AssertionError
xiaorui created CALCITE-2315: Summary: Inner-join query failed : Java.lang.AssertionError Key: CALCITE-2315 URL: https://issues.apache.org/jira/browse/CALCITE-2315 Project: Calcite Issue Type: Bug Reporter: xiaorui Assignee: Julian Hyde Attachments: logs "select table1.\"int1\",table1.\"str1\",table2.\"int1\",table2.\"str1\" from table1 inner join table2 on table1.\"id\" = table2.\"id\" where table1.\"id\" = 1" AssertionError would be thrown out When I try to run the above SQL ,The stack is as following (Detailed log see attachment). java.lang.AssertionError at org.apache.calcite.adapter.elasticsearch.ElasticsearchProject.(ElasticsearchProject.java:44) at org.apache.calcite.adapter.elasticsearch.ElasticsearchProject.copy(ElasticsearchProject.java:49) at org.apache.calcite.rel.rules.FilterProjectTransposeRule.onMatch(FilterProjectTransposeRule.java:131) at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212) at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:650) at org.apache.calcite.tools.Programs$5.run(Programs.java:326) at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:387) at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:188) at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:319) at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:230) at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:781) at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640) at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610) at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:221) at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603) at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638) at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149) at org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:209) at org.apache.calcite.test.ElasticsearchAdapterTest.testElasticSearch(ElasticsearchAdapterTest.java:35) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-500) Ensure EnumerableJoin hashes the smallest input
[
https://issues.apache.org/jira/browse/CALCITE-500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479003#comment-16479003
]
Atri Sharma commented on CALCITE-500:
-
Ignore, somehow the tests did not break locally. I will fix the PR.
> Ensure EnumerableJoin hashes the smallest input
> ---
>
> Key: CALCITE-500
> URL: https://issues.apache.org/jira/browse/CALCITE-500
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.0.0-incubating
>Reporter: Vladimir Sitnikov
>Assignee: Atri Sharma
>Priority: Major
> Labels: newbie
> Attachments: lookup.png
>
>
> {{EnumerableJoin}} tries to put the smallest input the first, however when it
> comes to execution, Calcite creates lookup for _second_ input of join.
> It would be nice to ensure the lookup is created on the smallest input.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-500) Ensure EnumerableJoin hashes the smallest input
[
https://issues.apache.org/jira/browse/CALCITE-500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478903#comment-16478903
]
Atri Sharma commented on CALCITE-500:
-
PR:
[https://github.com/apache/calcite/pull/691]
[~julianhyde] [~vladimirsitnikov] Please take a look and let me know. I tested
and saw that for inner joins, the smaller side is being built and the larger
side is being probed.
Please see and let me know
> Ensure EnumerableJoin hashes the smallest input
> ---
>
> Key: CALCITE-500
> URL: https://issues.apache.org/jira/browse/CALCITE-500
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.0.0-incubating
>Reporter: Vladimir Sitnikov
>Assignee: Atri Sharma
>Priority: Major
> Labels: newbie
> Attachments: lookup.png
>
>
> {{EnumerableJoin}} tries to put the smallest input the first, however when it
> comes to execution, Calcite creates lookup for _second_ input of join.
> It would be nice to ensure the lookup is created on the smallest input.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2312) Support Partition By in sql select statement
[
https://issues.apache.org/jira/browse/CALCITE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478405#comment-16478405
]
Fei Xu edited comment on CALCITE-2312 at 5/17/18 9:41 AM:
--
Thanks for the comment.
So, may be I can start from SqlCreateTable, add PARTITION BY strategy, just
like adding PRIMARY KEY. SqlCreateTable represents a relation table, maybe
SqlCreateStream is more suitable for stream.
was (Author: xu fei):
Thanks for the comment.
So, may be I can start from SqlCreateTable, add PARTITION BY strategy, just
like adding PRIMARY KEY.
> Support Partition By in sql select statement
>
>
> Key: CALCITE-2312
> URL: https://issues.apache.org/jira/browse/CALCITE-2312
> Project: Calcite
> Issue Type: New Feature
> Components: core
>Reporter: Fei Xu
>Assignee: Julian Hyde
>Priority: Major
>
> I noticed that calcite already have an Exchange RelNode represents
> distribution on RelNode's input, but no sql clause support.
>
> We are use calcite building SQL layer for out streaming platform. and in
> streaming computation, data shuffle is a very important function, not only
> for our engine, but also for our users.
> For example, In stream-join-table, the engine will load the table data to a
> cache at runtime, and stream join table is actually stream look up table.
> If the stream data could partition by hash or range, before look up table. It
> will be cache friendly cause particular stream data look up particular table
> data.
>
> So I consider do some extensions on sql clause to support Exchange, e.g.
> {code:java}
> // shuffle data using hash_distribution
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY o.productId
> // shuffle data to a singleton node
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY SINGLETON
> // shuffle data to all nodes
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY BROADCAST
> // shuffle data to random node
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY RANDOM
> // shuffle data using round-robin policy
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY ROUND_ROBIN
> // shuffle data using range policy
> // Current I'm not sure about the appropriate clause to represents range
> // shuffle, so it is just an demo.
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY RANGE o.productId, 0, 4096
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2313) Showing Empty Result
[ https://issues.apache.org/jira/browse/CALCITE-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478611#comment-16478611 ] Subbarao edited comment on CALCITE-2313 at 5/17/18 6:50 AM: how to get all function names and procedure names in oracle database using calcite connection and schema. schema.getFunctionNames() return result is:[ ] Then i am try to retrieving DatabaseMetadata.getFunction().Here also it will be showing empty. Iam using calciResultSet ,AvaticaResultSet and ResultSet for getting functions and procedures but will be showing empty. Is it possible get all function names and procedure names from any database. But normal jdbc connection using DatabseMetaData properties it retrive alll function names and procedure names.Why it will not retrive in calcite was (Author: subbumitta): how to get all function names and procedure names in oracle database using calcite connection and schema. schema.getFunctionNames() return result is:[ ] Then i am try to retrieving DatabaseMetadata.getFunction().Here also it will be showing empty. Iam using calciResultSet ,AvaticaResultSet and ResultSet for getting functions and procedures but will be showing empty. Is it possible get all function names and procedure names from any database > Showing Empty Result > > > Key: CALCITE-2313 > URL: https://issues.apache.org/jira/browse/CALCITE-2313 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.15.0 >Reporter: Subbarao >Assignee: Julian Hyde >Priority: Major > > While i am executing getFunctionNames() in Schema it will be showing Empty [ ] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2313) Showing Empty Result
[ https://issues.apache.org/jira/browse/CALCITE-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478611#comment-16478611 ] Subbarao commented on CALCITE-2313: --- how to get all function names and procedure names in oracle database using calcite connection and schema. schema.getFunctionNames() return result is:[ ] Then i am try to retrieving DatabaseMetadata.getFunction().Here also it will be showing empty. Iam using calciResultSet ,AvaticaResultSet and ResultSet for getting functions and procedures but will be showing empty. Is it possible get all function names and procedure names from any database > Showing Empty Result > > > Key: CALCITE-2313 > URL: https://issues.apache.org/jira/browse/CALCITE-2313 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.15.0 >Reporter: Subbarao >Assignee: Julian Hyde >Priority: Major > > While i am executing getFunctionNames() in Schema it will be showing Empty [ ] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
