[jira] [Created] (CALCITE-3061) Query with WITH clause fails when alias is the same as the table with rolled up column

2019-05-10 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-3061:


 Summary: Query with WITH clause fails when alias is the same as 
the table with rolled up column
 Key: CALCITE-3061
 URL: https://issues.apache.org/jira/browse/CALCITE-3061
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Volodymyr Vysotskyi
 Fix For: 1.20.0


{{SqlValidatorImpl}} incorrectly resolves aliases for CTE when checking rolled 
up columns when a table with the same name is present.

It may be observed with the following case: we have table {{emp_r}} which 
contains rolled up column {{slackingmin}}. The following query:
{code:sql}
with emp_r as (select 1 as slackingmin) select slackingmin from emp_r
{code}
should pass since it uses CTE, not table.
 Error for this case:
{noformat}
org.apache.calcite.sql.validate.SqlValidatorException: Rolled up column 
'SLACKINGMIN' is not allowed in SELECT
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at 
org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:787)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:772)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4825)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3438)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3447)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUpInSelectList(SqlValidatorImpl.java:3373)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3361)
at 
org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
at 
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:994)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:954)
at 
org.apache.calcite.sql.validate.WithNamespace.validateImpl(WithNamespace.java:57)
at 
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:994)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateWith(SqlValidatorImpl.java:3787)
at org.apache.calcite.sql.SqlWith.validate(SqlWith.java:71)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:929)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:633)
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-2745) RexSimplify fails with ClassCastException when the expression contains comparisons of operands with different types

2018-12-17 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2745:


 Summary: RexSimplify fails with ClassCastException when the 
expression contains comparisons of operands with different types
 Key: CALCITE-2745
 URL: https://issues.apache.org/jira/browse/CALCITE-2745
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


{{RexSimplify}} fails with {{ClassCastException}} when the expression contains 
predicates with comparisons of operands with different types.

Test to reproduce this issue (placed into {{JdbcAdapterTest}}):
{code:sql}
  @Test public void testSeveralTypesComparison() {
CalciteAssert.model(JdbcTest.FOODMART_MODEL)
.query("SELECT \"full_name\" FROM \"employee\" WHERE "
+ "\"employee_id\" = '1' and \"employee_id\" > 0")
.returns("full_name=Sheri Nowmer\n");
  }
{code}
It fails with the error:
{noformat}
Caused by: java.lang.ClassCastException: org.apache.calcite.util.NlsString 
cannot be cast to java.math.BigDecimal
at java.math.BigDecimal.compareTo(BigDecimal.java:220)
at 
org.apache.calcite.rex.RexSimplify.processRange(RexSimplify.java:1780)
at 
org.apache.calcite.rex.RexSimplify.simplifyAnd2ForUnknownAsFalse(RexSimplify.java:1242)
at 
org.apache.calcite.rex.RexSimplify.simplifyAnd2ForUnknownAsFalse(RexSimplify.java:1109)
at org.apache.calcite.rex.RexSimplify.simplifyAnds(RexSimplify.java:386)
at 
org.apache.calcite.rex.RexSimplify.simplifyFilterPredicates(RexSimplify.java:2028)
at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1048)
at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1037)
at 
org.apache.calcite.rel.rules.PushProjector.convertProject(PushProjector.java:382)
at 
org.apache.calcite.rel.rules.ProjectFilterTransposeRule.onMatch(ProjectFilterTransposeRule.java:104)
at 
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
... 41 more
{noformat}
When the query has any of the {{AND}} operands, or when 
{{ProjectFilterTransposeRule}} is not applied it is passed.

Regarding the comparison of different data types, SQL spec says the following:
{quote}8.2  Syntax Rules
 3) The declared types of the corresponding fields of the two s shall be comparable.
{quote}
But it also allows implicit casts from numeric types to chars (6.13 ).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-2497) Update Janino version to 3.0.9

2018-08-28 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2497:


 Summary: Update Janino version to 3.0.9
 Key: CALCITE-2497
 URL: https://issues.apache.org/jira/browse/CALCITE-2497
 Project: Calcite
  Issue Type: Task
Affects Versions: 1.17.0
Reporter: Volodymyr Vysotskyi
Assignee: Volodymyr Vysotskyi
 Fix For: 1.18.0


Update Janino version to 3.0.9 where was fixed 
[https://github.com/janino-compiler/janino/issues/47], and remove workaround 
made in CALCITE-2261 because of this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-2422) Query with unnest of column from nested subquery fails when dynamic table is used

2018-07-20 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2422:


 Summary: Query with unnest of column from nested subquery fails 
when dynamic table is used
 Key: CALCITE-2422
 URL: https://issues.apache.org/jira/browse/CALCITE-2422
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


This test:
{code:java}
  @Test public void testStarDynamicSchemaUnnestNestedSubquery() {
String sql3 = "select * \n"
+ "from (select * from SALES.CUSTOMER) as t1,\n"
+ "unnest(t1.fake_col) as t2";
sql(sql3).with(getTesterWithDynamicTable()).ok();
  }
{code}
placed in {{SqlToRelConverterTest}} fails with {{AssertionError}}:
{noformat}
java.lang.AssertionError
at 
org.apache.calcite.sql.SqlUnnestOperator.inferReturnType(SqlUnnestOperator.java:81)
at 
org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:437)
at 
org.apache.calcite.sql.validate.UnnestNamespace.validateImpl(UnnestNamespace.java:67)
at 
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:968)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:944)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:3032)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:3017)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateJoin(SqlValidatorImpl.java:3069)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:3026)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3275)
at 
org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
at 
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:968)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:944)
at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:226)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:919)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:629)
at 
org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:642)
at 
org.apache.calcite.test.SqlToRelTestBase$TesterImpl.assertConvertsTo(SqlToRelTestBase.java:754)
at 
org.apache.calcite.test.SqlToRelConverterTest$Sql.convertsTo(SqlToRelConverterTest.java:2905)
at 
org.apache.calcite.test.SqlToRelConverterTest$Sql.ok(SqlToRelConverterTest.java:2897)
at 
org.apache.calcite.test.SqlToRelConverterTest.testStarDynamicSchemaUnnestNestedSubquery(SqlToRelConverterTest.java:2530)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at 
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at 
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at 
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at 

[jira] [Created] (CALCITE-2415) Evaluation of predicate "(A or B) and C" is failing for MongoDB adapter

2018-07-15 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2415:


 Summary: Evaluation of predicate "(A or B) and C" is failing for 
MongoDB adapter
 Key: CALCITE-2415
 URL: https://issues.apache.org/jira/browse/CALCITE-2415
 Project: Calcite
  Issue Type: Bug
  Components: elasticsearch-adapter, geode, mongodb
Reporter: Andrei Sereda
Assignee: Michael Mior


h2. Queries that are working
{code}
select count(*) from "elastic" where _MAP['foo'] in ('1') and true;
select count(*) from "elastic" where _MAP['foo'] ='1' and true;
select count(*) from "elastic" where _MAP['foo'] in ('1', '2');
{code}

h2. Queries that are failing
{code}
select count(*) from "elastic" where _MAP['foo'] in ('1', '2') and true;
select count(*) from "elastic" where true and _MAP['foo'] in ('1', '2');
select count(*) from "elastic" where (_MAP['foo'] ='1' or _MAP['foo'] = '2') 
and true;
{code}

h2. Elastic Adapter Exception
{quote}
java.lang.AssertionError: cannot translate OR(=(ITEM($0, 'foo'), '1'),
=(ITEM($0, 'foo'), '2'))
at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
Translator.translateMatch2(ElasticsearchFilter.java:234)
at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
Translator.translateAnd(ElasticsearchFilter.java:158)
at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
Translator.translateOr(ElasticsearchFilter.java:115)
at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
Translator.translateMatch(ElasticsearchFilter.java:101)
at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
Translator.access$000(ElasticsearchFilter.java:87)
at org.apache.calcite.adapter.elasticsearch.
ElasticsearchFilter.implement(ElasticsearchFilter.java:80)
at org.apache.calcite.adapter.elasticsearch.
ElasticsearchRel$Implementor.visitChild(ElasticsearchRel.java:53)
at org.apache.calcite.adapter.elasticsearch.
ElasticsearchToEnumerableConverter.implement(ElasticsearchToEnumerableConve
rter.java:71)
at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.
visitChild(EnumerableRelImplementor.java:98)
at org.apache.calcite.adapter.enumerable.
EnumerableAggregate.implement(EnumerableAggregate.java:106)
{quote}

h2. Mongo Adapter Exception
{quote}
select * from "mongo"."col"
where _MAP['foo'] in ('1', '2') and true;

java.lang.AssertionError: cannot translate OR(=(ITEM($0, 'foo'), '1'),
=(ITEM($0, 'foo'), '2'))
at org.apache.calcite.adapter.mongodb.MongoFilter$
Translator.translateMatch2(MongoFilter.java:181)
at org.apache.calcite.adapter.mongodb.MongoFilter$
Translator.translateAnd(MongoFilter.java:116)
at org.apache.calcite.adapter.mongodb.MongoFilter$
Translator.translateOr(MongoFilter.java:98)
at org.apache.calcite.adapter.mongodb.MongoFilter$
Translator.translateMatch(MongoFilter.java:91)
at org.apache.calcite.adapter.mongodb.MongoFilter$
Translator.access$000(MongoFilter.java:77)
at org.apache.calcite.adapter.mongodb.MongoFilter.implement(
MongoFilter.java:72)
at org.apache.calcite.adapter.mongodb.MongoRel$Implementor.
visitChild(MongoRel.java:51)
at org.apache.calcite.adapter.mongodb.MongoToEnumerableConverter.
implement(MongoToEnumerableConverter.java:84)
at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.
implementRoot(EnumerableRelImplementor.java:103)
at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.
toBindable(EnumerableInterpretable.java:92)
{quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-2400) Allow standards-compliant column ordering for NATURAL JOIN and JOIN USING when dynamic tables are used

2018-07-04 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2400:


 Summary: Allow standards-compliant column ordering for NATURAL 
JOIN and JOIN USING when dynamic tables are used
 Key: CALCITE-2400
 URL: https://issues.apache.org/jira/browse/CALCITE-2400
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


In CALCITE-2227 was allowed standards-compliant column ordering for NATURAL 
JOIN and JOIN USING for non-dynamic tables. In CALCITE-2392 was prevented 
permutation of columns for dynamic tables to avoid IOBE.

The aim of this Jira is to implement standards-compliant column ordering for 
NATURAL JOIN and JOIN USING for dynamic tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-2391) Query with aggregation of UNNEST column fails with ClassCastException

2018-07-01 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2391:


 Summary: Query with aggregation of UNNEST column fails with 
ClassCastException
 Key: CALCITE-2391
 URL: https://issues.apache.org/jira/browse/CALCITE-2391
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


This test:
{code:java}
  @Test public void testAggUnnestColumn() {
CalciteAssert.hr()
.query("select count(d.\"name\") as c\n"
+ "from \"hr\".\"depts\" as d,\n"
+ " UNNEST(d.\"employees\") as e")
.returnsUnordered(
"C=3");
  }
{code}
added to {{JdbcTest}} class fails with {{ClassCastException}}
{noformat}
Caused by: java.lang.ClassCastException: 
org.apache.calcite.sql.validate.AggregatingSelectScope cannot be cast to 
org.apache.calcite.sql.validate.ListScope
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.lookupExp(SqlToRelConverter.java:4293)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3614)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.access$2300(SqlToRelConverter.java:213)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4692)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3990)
at org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:344)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4554)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2090)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:1957)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2033)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:642)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:623)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3058)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:559)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:264)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:230)
at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:783)
at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:642)
at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:612)
at 
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:232)
at 
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:609)
at 
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
at 
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
... 28 more
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-2389) Drill query fails when file name contains semicolon

2018-06-29 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2389:


 Summary: Drill query fails when file name contains semicolon
 Key: CALCITE-2389
 URL: https://issues.apache.org/jira/browse/CALCITE-2389
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


Queries on the tables which contain semicolon in the name:
{code:sql}
select * from dfs.`/tmp/q:0`
{code}
fails with error:
{noformat}
org.apache.drill.common.exceptions.UserRemoteException: VALIDATION ERROR: 
java.net.URISyntaxException: Relative path in absolute URI: q:0

SQL Query null

[Error Id: 34fafee1-8fbe-4fe0-9fcb-ddcc926bb192 on user515050-pc:31010]

  (java.lang.IllegalArgumentException) java.net.URISyntaxException: Relative 
path in absolute URI: q:0
org.apache.hadoop.fs.Path.initialize():205
org.apache.hadoop.fs.Path.():171
org.apache.hadoop.fs.Path.():93
org.apache.hadoop.fs.Globber.glob():253
org.apache.hadoop.fs.FileSystem.globStatus():1655
org.apache.drill.exec.store.dfs.DrillFileSystem.globStatus():547
org.apache.drill.exec.store.dfs.FileSelection.create():274

org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create():607

org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create():408
org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry():96
org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.get():90

org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.getTable():561

org.apache.drill.exec.store.dfs.FileSystemSchemaFactory$FileSystemSchema.getTable():132
org.apache.calcite.jdbc.SimpleCalciteSchema.getImplicitTable():82
org.apache.calcite.jdbc.CalciteSchema.getTable():257
org.apache.calcite.sql.validate.SqlValidatorUtil.getTableEntryFrom():1022
org.apache.calcite.sql.validate.SqlValidatorUtil.getTableEntry():979
org.apache.calcite.prepare.CalciteCatalogReader.getTable():123

org.apache.drill.exec.planner.sql.SqlConverter$DrillCalciteCatalogReader.getTable():650

org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():260
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3219
org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60
org.apache.calcite.sql.validate.AbstractNamespace.validate():84
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():947
org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():928
org.apache.calcite.sql.SqlSelect.validate():226

org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():903
org.apache.calcite.sql.validate.SqlValidatorImpl.validate():613
org.apache.drill.exec.planner.sql.SqlConverter.validate():190

org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode():644

org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert():204
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():176
org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83
org.apache.drill.exec.work.foreman.Foreman.runSQL():567
org.apache.drill.exec.work.foreman.Foreman.run():266
java.util.concurrent.ThreadPoolExecutor.runWorker():1149
java.util.concurrent.ThreadPoolExecutor$Worker.run():624
java.lang.Thread.run():748
  Caused By (java.net.URISyntaxException) Relative path in absolute URI: q:0
java.net.URI.checkPath():1823
java.net.URI.():745
org.apache.hadoop.fs.Path.initialize():202
org.apache.hadoop.fs.Path.():171
org.apache.hadoop.fs.Path.():93
org.apache.hadoop.fs.Globber.glob():253
org.apache.hadoop.fs.FileSystem.globStatus():1655
org.apache.drill.exec.store.dfs.DrillFileSystem.globStatus():547
org.apache.drill.exec.store.dfs.FileSelection.create():274

org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create():607

org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create():408
org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry():96
org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.get():90

org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.getTable():561

org.apache.drill.exec.store.dfs.FileSystemSchemaFactory$FileSystemSchema.getTable():132
org.apache.calcite.jdbc.SimpleCalciteSchema.getImplicitTable():82
org.apache.calcite.jdbc.CalciteSchema.getTable():257
org.apache.calcite.sql.validate.SqlValidatorUtil.getTableEntryFrom():1022
org.apache.calcite.sql.validate.SqlValidatorUtil.getTableEntry():979
org.apache.calcite.prepare.CalciteCatalogReader.getTable():123

org.apache.drill.exec.planner.sql.SqlConverter$DrillCalciteCatalogReader.getTable():650


[jira] [Created] (CALCITE-2379) CVSS dependency-check-maven fails for calcite-spark module

2018-06-25 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2379:


 Summary: CVSS dependency-check-maven fails for calcite-spark module
 Key: CALCITE-2379
 URL: https://issues.apache.org/jira/browse/CALCITE-2379
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


Check for vulnerabilities among dependencies fails for {{calcite-spark}} module.

Output for "{{mvn install -Ppedantic -DskipTests=true}}":
{noformat}
One or more dependencies were identified with known vulnerabilities in Calcite 
Spark:

jackson-databind-2.9.4.jar (com.fasterxml.jackson.core:jackson-databind:2.9.4, 
cpe:/a:fasterxml:jackson-databind:2.9.4, cpe:/a:fasterxml:jackson:2.9.4) : 
CVE-2018-7489
protobuf-java-3.3.0.jar (com.google.protobuf:protobuf-java:3.3.0, 
cpe:/a:google:protobuf:3.3.0) : CVE-2015-5237
commons-beanutils-core-1.8.0.jar 
(commons-beanutils:commons-beanutils-core:1.8.0, 
cpe:/a:apache:commons_beanutils:1.8.0) : CVE-2014-0114
commons-beanutils-1.7.0.jar (commons-beanutils:commons-beanutils:1.7.0, 
cpe:/a:apache:commons_beanutils:1.7.0) : CVE-2014-0114
commons-httpclient-3.1.jar (commons-httpclient:commons-httpclient:3.1, 
cpe:/a:apache:commons-httpclient:3.1, cpe:/a:apache:httpclient:3.1) : 
CVE-2015-5262, CVE-2014-3577
javax.annotation-api-1.2.jar (cpe:/a:oracle:glassfish:1.2, 
javax.annotation:javax.annotation-api:1.2) : CVE-2015-2808, CVE-2013-2566
mail-1.4.7.jar (cpe:/a:mail_project:mail:1.4.7, javax.mail:mail:1.4.7) : 
CVE-2015-9097
validation-api-1.1.0.Final.jar (cpe:/a:bean_project:bean:7.x-1.1::~~~drupal~~, 
javax.validation:validation-api:1.1.0.Final) : CVE-2013-4499
jaxb-api-2.2.2.jar (cpe:/a:fish:fish:2.2.2, cpe:/a:oracle:glassfish:2.2.2, 
javax.xml.bind:jaxb-api:2.2.2) : CVE-2015-2808, CVE-2013-2566
pyrolite-4.13.jar (cpe:/a:pickle:pickle:4.13, net.razorvine:pyrolite:4.13) : 
CVE-2007-1100
py4j-0.10.4.jar (cpe:/a:python:python:0.10.4, 
cpe:/a:python_software_foundation:python:0.10.4, net.sf.py4j:py4j:0.10.4) : 
CVE-2018-130, CVE-2017-18207, CVE-2017-17522, CVE-2017-1000158, 
CVE-2016-5699, CVE-2016-5636, CVE-2016-1494, CVE-2016-0772, CVE-2015-5652, 
CVE-2014-7185, CVE-2014-3539, CVE-2013-7440, CVE-2013-7338, CVE-2012-1150, 
CVE-2012-0845, CVE-2011-4940, CVE-2010-3492, CVE-2008-5983, CVE-2008-3143, 
CVE-2008-3142, CVE-2008-2315, CVE-2008-1887, CVE-2008-1721, CVE-2008-1679, 
CVE-2007-4559, CVE-2006-1542, CVE-2002-1119
avro-mapred-1.7.7-hadoop2.jar (cpe:/a:apache:hadoop:1.7.7, 
org.apache.avro:avro-mapred:1.7.7) : CVE-2017-3162, CVE-2017-3161, CVE-2016-5001
curator-recipes-2.6.0.jar (cpe:/a:apache:zookeeper:2.6.0, 
org.apache.curator:curator-recipes:2.6.0) : CVE-2016-5017, CVE-2014-0085
api-util-1.0.0-M20.jar (cpe:/a:apache:directory_ldap_api:1.0.0.m30, 
org.apache.directory.api:api-util:1.0.0-M20) : CVE-2015-3250
xbean-asm5-shaded-4.4.jar (cpe:/a:apache:geronimo:4.4) : CVE-2008-0732
zookeeper-3.4.6.jar (cpe:/a:apache:zookeeper:3.4.6, 
org.apache.zookeeper:zookeeper:3.4.6) : CVE-2017-5637, CVE-2016-5017, 
CVE-2014-0085
jackson-xc-1.9.13.jar (cpe:/a:fasterxml:jackson-databind:1.9.13, 
cpe:/a:fasterxml:jackson:1.9.13, org.codehaus.jackson:jackson-xc:1.9.13) : 
CVE-2018-5968, CVE-2017-17485
jetty-http-9.2.19.v20160908.jar (cpe:/a:eclipse:jetty:9.2.19.v20160908, 
cpe:/a:jetty:jetty:9.2.19.v20160908, 
org.eclipse.jetty:jetty-http:9.2.19.v20160908) : CVE-2017-9735
jetty-util-6.1.26.jar (cpe:/a:jetty:jetty:6.1.26, cpe:/a:mortbay:jetty:6.1.26, 
cpe:/a:mortbay_jetty:jetty:6.1.26, org.mortbay.jetty:jetty-util:6.1.26) : 
CVE-2011-4461
unused-1.0.0.jar (cpe:/a:apache:spark:1.0.0, 
org.spark-project.spark:unused:1.0.0) : CVE-2017-7678
xz-1.0.jar (cpe:/a:tukaani:xz:1.0, org.tukaani:xz:1.0) : CVE-2015-4035
serializer-2.7.1.jar (cpe:/a:apache:xalan-java:2.7.1, xalan:serializer:2.7.1) : 
CVE-2014-0107
xalan-2.7.1.jar (cpe:/a:apache:xalan-java:2.7.1, xalan:xalan:2.7.1) : 
CVE-2014-0107
xercesImpl-2.9.1.jar (cpe:/a:apache:xerces2_java:2.9.1, 
xerces:xercesImpl:2.9.1) : CVE-2012-0881
htrace-core-3.1.0-incubating.jar/META-INF/maven/com.fasterxml.jackson.core/jackson-databind/pom.xml
 (com.fasterxml.jackson.core:jackson-databind:2.4.0, 
cpe:/a:fasterxml:jackson-databind:2.4.0, cpe:/a:fasterxml:jackson:2.4.0) : 
CVE-2018-7489, CVE-2018-5968, CVE-2017-7525, CVE-2017-17485, CVE-2017-15095
spark-core_2.10-2.2.0.jar/META-INF/maven/org.eclipse.jetty/jetty-plus/pom.xml 
(cpe:/a:eclipse:jetty:9.3.11.v20160721, cpe:/a:jetty:jetty:9.3.11.v20160721, 
org.eclipse.jetty:jetty-plus:9.3.11.v20160721) : CVE-2017-9735
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-2368) Fix misc.iq and scalar.iq quidem unit tests failures on Windows

2018-06-16 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2368:


 Summary: Fix misc.iq and scalar.iq quidem unit tests failures on 
Windows
 Key: CALCITE-2368
 URL: https://issues.apache.org/jira/browse/CALCITE-2368
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


misc.iq and scalar.iq quidem unit tests fail on Windows but pass on Linux.

*Problem description*
Files with quidem tests are discovered in runtime. Some tests, such as misc.iq 
or scalar.iq require setting up {{THREAD_EXPAND}} to true (see 
{{CoreQuidemTest.testSqlMisc()}} and {{CoreQuidemTest.testSqlScalar()}} 
methods).

Methods, which should be called for concrete quidem tests to set required 
variables are discovered using the filename.

Code replaces character {{'/'}} in the filepath, but in the case of Windows in 
the path was path separator char for Windows.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-2337) Release Calcite 1.17.0

2018-05-30 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2337:


 Summary: Release Calcite 1.17.0
 Key: CALCITE-2337
 URL: https://issues.apache.org/jira/browse/CALCITE-2337
 Project: Calcite
  Issue Type: Task
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde
 Fix For: 1.17.0


Release Calcite 1.17.0



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-2329) Enhance SubQueryRemoveRule to rewrite IN operator with the constant from the left side more optimally

2018-05-25 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2329:


 Summary: Enhance SubQueryRemoveRule to rewrite IN operator with 
the constant from the left side more optimally
 Key: CALCITE-2329
 URL: https://issues.apache.org/jira/browse/CALCITE-2329
 Project: Calcite
  Issue Type: Improvement
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


Currently, for queries like this:
{code:sql}
select sal from emp where 2 IN (select deptno from dept)
{code}
SubQueryRemoveRule rules expand query plan in such a way:
{noformat}
LogicalProject(SAL=[$5])
  LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], 
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
LogicalJoin(condition=[=(2, $9)], joinType=[inner])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
  LogicalProject(DEPTNO=[$0])
LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
{noformat}
Since join condition is actually a filter condition, it will be pushed into the 
filter during further planning stages and join with the true condition is left.

But these types of the queries may be rewritten in the same way as EXISTS 
queries:
{code:sql}
select sal from emp where EXISTS (select deptno from dept where deptno=2)
{code}
with the more optimal plan:
{noformat}
LogicalProject(SAL=[$5])
  LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], 
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
LogicalJoin(condition=[true], joinType=[inner])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
  LogicalAggregate(group=[{0}])
LogicalProject(i=[true])
  LogicalFilter(condition=[=($0, 2)])
LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-2223) ProjectMergeRule is infinitely matched when is applied after ProjectReduceExpressionsRule

2018-03-25 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2223:


 Summary: ProjectMergeRule is infinitely matched when is applied 
after ProjectReduceExpressionsRule
 Key: CALCITE-2223
 URL: https://issues.apache.org/jira/browse/CALCITE-2223
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


For queries like this:
{code:sql}
select t1.f from (select cast(f as int) f, f from (select cast(f as int) f from 
(values('1')) t(f))) as t1
{code}
OOM is thrown when {{ProjectMergeRule}} is applied before applying 
{{ProjectReduceExpressionsRule}} in VolcanoPlanner.
 A simple test to reproduce this issue:
{code:java}
  @Test public void testOomProjectMergeRule() {
RelBuilder relBuilder = RelBuilder.create(RelBuilderTest.config().build());
RelNode relNode = relBuilder
.values(new String[]{"f"}, "1")
.project(
relBuilder.alias(
relBuilder.cast(relBuilder.field(0), SqlTypeName.INTEGER),
"f"))
.project(
relBuilder.alias(
relBuilder.cast(relBuilder.field(0), SqlTypeName.INTEGER),
"f0"),
relBuilder.alias(relBuilder.field(0), "f"))
.project(
relBuilder.alias(relBuilder.field(0), "f"))
.build();

RelOptPlanner planner = relNode.getCluster().getPlanner();
RuleSet ruleSet =
RuleSets.ofList(
ReduceExpressionsRule.PROJECT_INSTANCE,
new ProjectMergeRuleWithLongerName(),
EnumerableRules.ENUMERABLE_PROJECT_RULE,
EnumerableRules.ENUMERABLE_VALUES_RULE);
Program program = Programs.of(ruleSet);

RelTraitSet toTraits =
relNode.getCluster().traitSet()
.replace(0, EnumerableConvention.INSTANCE);

RelNode output = program.run(planner, relNode, toTraits,
ImmutableList.of(), 
ImmutableList.of());

// check for output
  }

  /**
   * ProjectMergeRule inheritor which has
   * class name longer than ProjectReduceExpressionsRule class name.
   *
   * It is needed for RuleQueue.popMatch() method
   * to apply this rule before ProjectReduceExpressionsRule.
   */
  private static class ProjectMergeRuleWithLongerName extends ProjectMergeRule {
public ProjectMergeRuleWithLongerName() {
  super(true, RelFactories.LOGICAL_BUILDER);
}
  }
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-2215) Avatica - Remove unused shaded jars

2018-03-16 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2215:


 Summary: Avatica - Remove unused shaded jars
 Key: CALCITE-2215
 URL: https://issues.apache.org/jira/browse/CALCITE-2215
 Project: Calcite
  Issue Type: Task
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


Currently shaded/core/pom.xml, standalone-server/pom.xml and tck/pom.xml pom 
files shade jars which are not used anywhere in the project:
- com.google.common
- org.apache.http
- org.apache.commons

It may cause problems for projects which shade Avatica and have dependencies to 
specified libraries.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-2201) Allow passing custom RelBuilder into RelDecorrelator and RelStructuredTypeFlattener

2018-03-03 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2201:


 Summary: Allow passing custom RelBuilder into RelDecorrelator and 
RelStructuredTypeFlattener
 Key: CALCITE-2201
 URL: https://issues.apache.org/jira/browse/CALCITE-2201
 Project: Calcite
  Issue Type: Task
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


When {{RelDecorrelator.decorrelateQuery()}} method is called, 
{{RelDecorrelator}} instance with {{RelFactories.LOGICAL_BUILDER}} is created. 
We should modify this method to allow usage of custom {{RelBuilder}}.

Also, {{RelStructuredTypeFlattener}} class should be modified in a similar way.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-2166) Cumulative cost of RelSubset.best RelNode is increased after calling RelSubset.propagateCostImprovements() for input RelNodes

2018-02-05 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2166:


 Summary: Cumulative cost of RelSubset.best RelNode is increased 
after calling RelSubset.propagateCostImprovements() for input RelNodes
 Key: CALCITE-2166
 URL: https://issues.apache.org/jira/browse/CALCITE-2166
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.15.0
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


After calling {{RelSubset.propagateCostImprovements()}} cumulative cost of 
{{RelSubset.best}} {{RelNode}} may be increased due to the increase of the 
non-cumulative cost caused by changing of input best {{RelNode}}.
To observe this issue, add this code:
{code:java}
  if (subset.best != null) {
RelOptCost bestCost = getCost(subset.best, 
RelMetadataQuery.instance());
if (!subset.bestCost.equals(bestCost)) {
  throw new AssertionError(
"relSubset [" + subset.getDescription()
  + "] has wrong best cost "
  + subset.bestCost + ". Correct cost is " + bestCost);
}
  }
{code}
into {{VolcanoPlanner.validate()}} method (line 907).
List of unit tests which fail with this check:
{noformat}
Failed tests: 
  
MaterializationTest.testJoinMaterializationUKFK9:1823->checkMaterialize:198->checkMaterialize:205->checkThatMaterialize:233
 relSubset [rel#226287:Subset#8.ENUMERABLE.[]] has wrong best cost {221.5 rows, 
128.25 cpu, 0.0 io}. Correct cost is {233.0 rows, 178.0 cpu, 0.0 io}
  ScannableTableTest.testPFPushDownProjectFilterAggregateNested:279 relSubset 
[rel#12950:Subset#5.ENUMERABLE.[]] has wrong best cost {63.8 rows, 62.308 cpu, 
0.0 io}. Correct cost is {70.4 rows, 60.404 cpu, 0.0 io}
  ScannableTableTest.testPFTableRefusesFilterCooperative:221 relSubset 
[rel#13382:Subset#2.ENUMERABLE.[]] has wrong best cost {81.0 rows, 181.01 cpu, 
0.0 io}. Correct cost is {150.5 rows, 250.505 cpu, 0.0 io}
  ScannableTableTest.testProjectableFilterableCooperative:148 relSubset 
[rel#13611:Subset#2.ENUMERABLE.[]] has wrong best cost {81.0 rows, 181.01 cpu, 
0.0 io}. Correct cost is {150.5 rows, 250.505 cpu, 0.0 io}
  ScannableTableTest.testProjectableFilterableNonCooperative:165 relSubset 
[rel#13754:Subset#2.ENUMERABLE.[]] has wrong best cost {81.0 rows, 181.01 cpu, 
0.0 io}. Correct cost is {150.5 rows, 250.505 cpu, 0.0 io}
  FrameworksTest.testUpdate:336->executeQuery:367 relSubset 
[rel#22533:Subset#2.ENUMERABLE.any] has wrong best cost {19.5 rows, 37.75 cpu, 
0.0 io}. Correct cost is {22.575 rows, 52.58 cpu, 0.0 io}
{noformat}
For the test {{MaterializationTest.testJoinMaterializationUKFK9}} initial best 
plan was:
{noformat}
EnumerableProject(empid0=[$5], empid00=[$5], deptno0=[$7]): rowcount = 15.0, 
cumulative cost = {15.0 rows, 45.0 cpu, 0.0 io}, id = 3989
  EnumerableJoin(subset=[rel#3988:Subset#34.ENUMERABLE.[]], condition=[=($1, 
$7)], joinType=[inner]): rowcount = 15.0, cumulative cost = {116.0 rows, 0.0 
cpu, 0.0 io}, id = 4797
EnumerableFilter(subset=[rel#4274:Subset#47.ENUMERABLE.[0]], 
condition=[=(CAST($2):VARCHAR CHARACTER SET "ISO-8859-1" COLLATE 
"ISO-8859-1$en_US$primary", 'Bill')]): rowcount = 1.0, cumulative cost = {1.0 
rows, 1.0 cpu, 0.0 io}, id = 16522
  EnumerableTableScan(subset=[rel#158:Subset#11.ENUMERABLE.[0]], 
table=[[hr, m0]]): rowcount = 1.0, cumulative cost = {0.0 rows, 1.0 cpu, 0.0 
io}, id = 79
EnumerableTableScan(subset=[rel#115:Subset#5.ENUMERABLE.[]], table=[[hr, 
depts]]): rowcount = 100.0, cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io}, 
id = 62
{noformat}
Its cumulative cost is \{221.5 rows, 123.75 cpu, 0.0 io}

After applying some rules it became:
{noformat}
EnumerableProject(empid0=[$3], empid00=[$3], deptno0=[$0]): rowcount = 2.25, 
cumulative cost = {2.25 rows, 6.75 cpu, 0.0 io}, id = 4012
  EnumerableFilter(subset=[rel#4007:Subset#41.ENUMERABLE.[]], 
condition=[=(CAST($2):VARCHAR CHARACTER SET "ISO-8859-1" COLLATE 
"ISO-8859-1$en_US$primary", 'Bill')]): rowcount = 2.25, cumulative cost = {2.25 
rows, 15.0 cpu, 0.0 io}, id = 4811
EnumerableProject(subset=[rel#4203:Subset#61.ENUMERABLE.[]], deptno=[$7], 
deptno0=[$1], name0=[$2], empid0=[$5]): rowcount = 15.0, cumulative cost = 
{15.0 rows, 60.0 cpu, 0.0 io}, id = 4206
  EnumerableJoin(subset=[rel#4204:Subset#52.ENUMERABLE.[]], 
condition=[=($1, $7)], joinType=[inner]): rowcount = 15.0, cumulative cost = 
{116.0 rows, 0.0 cpu, 0.0 io}, id = 4795
EnumerableTableScan(subset=[rel#158:Subset#11.ENUMERABLE.[0]], 
table=[[hr, m0]]): rowcount = 1.0, cumulative cost = {0.0 rows, 1.0 cpu, 0.0 
io}, id = 79
EnumerableTableScan(subset=[rel#115:Subset#5.ENUMERABLE.[]], 
table=[[hr, depts]]): rowcount = 100.0, cumulative cost = {100.0 rows, 101.0 
cpu, 0.0 io}, id = 62
{noformat}
Its cumulative cost is {{{233.0 rows, 148.0 cpu, 0.0 io}}}. 

The new plan does not contain 

[jira] [Created] (CALCITE-2158) SubQuery with EXISTS clause creates redundant aggregate call

2018-01-30 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2158:


 Summary: SubQuery with EXISTS clause creates redundant aggregate 
call
 Key: CALCITE-2158
 URL: https://issues.apache.org/jira/browse/CALCITE-2158
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


When {{SqlToRelConverter.Config.isExpand()}} returns true, subqueries are 
expanded in {{SqlToRelConverter}}.
Then for the queries, like this:
{code:sql}
SELECT cs1.sal
FROM emp cs1
WHERE EXISTS
(SELECT *
 FROM emp cs2
 WHERE cs1.sal = cs2.sal
   AND cs1.deptno <> cs2.deptno)
{code}
Calcite returns logical plan with excessive aggregate calls:
{noformat}
LogicalProject(SAL=[$5])
  LogicalFilter(condition=[IS NOT NULL($9)])
LogicalCorrelate(correlation=[$cor0], joinType=[left], requiredColumns=[{5, 
7}])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
  LogicalAggregate(group=[{}], agg#0=[MIN($0)])
LogicalProject($f0=[true])
  LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], 
HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
LogicalFilter(condition=[AND(=($cor0.SAL, $5), <>($cor0.DEPTNO, 
$7))])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{noformat}
But when {{SqlToRelConverter.Config.isExpand()}} returns false and 
SubQueryRemoveRule rules are applied to the logical plan with RexSubQuery, the 
resulting logical plan is correct and does not contain excessive aggregate 
calls:
{noformat}
LogicalProject(SAL=[$5])
  LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], 
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
LogicalCorrelate(correlation=[$cor0], joinType=[inner], 
requiredColumns=[{5, 7}])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
  LogicalAggregate(group=[{0}])
LogicalProject(i=[true])
  LogicalFilter(condition=[AND(=($cor0.SAL, $5), <>($cor0.DEPTNO, $7))])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{noformat}
These cases may be observed using this unit test:
{code:java}
  @Test public void testExistsExpand() {
final HepProgram preProgram = HepProgram.builder()
.addRuleInstance(SubQueryRemoveRule.FILTER)
.addRuleInstance(SubQueryRemoveRule.PROJECT)
.addRuleInstance(SubQueryRemoveRule.JOIN)
.build();
final HepProgram program = HepProgram.builder()
.build();
final String sql = "SELECT cs1.sal\n"
+ "FROM emp cs1 \n" 
+ "WHEREEXISTS\n" 
+ "(SELECT *\n" 
+ "FROM   emp cs2\n" 
+ "WHERE  cs1.sal = cs2.sal\n" 
+ "ANDcs1.deptno <> cs2.deptno)";
sql(sql)
.withDecorrelation(false)
.withTrim(false)
.expand(true) // change to false
.withPre(preProgram)
.with(program)
.checkUnchanged();
  }
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-2092) Allow passing custom RelBuilder into SqlToRelConverter

2017-12-16 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2092:


 Summary: Allow passing custom RelBuilder into SqlToRelConverter
 Key: CALCITE-2092
 URL: https://issues.apache.org/jira/browse/CALCITE-2092
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


Currently, {{SqlToRelConverter}} uses {{RelBuilder}} which was created in such 
a way:
{code:java}
RelFactories.LOGICAL_BUILDER.create(cluster, null)
{code}
So it is not possible to use custom {{RelBuilder}} in this class.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (CALCITE-2087) Add new method to ViewExpander interface to allow passing SchemaPlus

2017-12-13 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2087:


 Summary: Add new method to ViewExpander interface to allow passing 
SchemaPlus
 Key: CALCITE-2087
 URL: https://issues.apache.org/jira/browse/CALCITE-2087
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


Currently, {{ViewExpander}} interface contains single method 
{{expandView(RelDataType rowType, String queryString, List schemaPath, 
List viewPath)}} which allows creating a {{RelNode}} instance that 
corresponds to the desired view.

Drill supports impersonation for views and it is implemented in such a way, 
that its configs such as username, etc are stored in {{SchemaPlus}} instance. 
So currently it is not possible to pass these configs into this method to 
create {{CatalogReader}} and allow impersonation work.





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (CALCITE-2071) Query with IN operators in WHERE clause returns wrong result

2017-11-29 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2071:


 Summary: Query with IN operators in WHERE clause returns wrong 
result
 Key: CALCITE-2071
 URL: https://issues.apache.org/jira/browse/CALCITE-2071
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


Query it this test returns the wrong result:
{code:java}
  @Test
  @Ignore
  public void testWhereInOr() throws Exception {
CalciteAssert.hr()
.query("select \"empid\"\n"
+ "from \"hr\".\"emps\" t\n"
+ "where (\"empid\" in (select \"empid\" from \"hr\".\"emps\") 
\n"
+ "or \"empid\" in (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, "
+ " 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 
23, 24, 25) )\n"
+ "  and \"empid\" in (100, 200, 150)")
.returns("empid=100\n"
+ "empid=200\n"
+ "empid=150\n");
  }
{code}
Without condition in or
{code:sql}
\"empid\" in (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 
19, 20, 21, 22, 23, 24, 25) 
{code}
it returns right result.

The problem appears during converting the query into the logical plan and may 
be observed more clearly using this test:
{code:java}
  @Test
  @Ignore
  public void testWhereInOrPlan() throws Exception {
final String sql =
"select \n" 
+ "from emp t\n" 
+ "where (t.EMPNO in (select EMPNO from emp) \n" 
+ "or t.EMPNO in (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, " 
+ "12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25) )\n" 
+ "  and t.EMPNO in (1, 2, 3)";
sql(sql).ok();
  }
{code}
This query returns plan 
{noformat}
LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], 
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
  LogicalFilter(condition=[AND(OR(true, true), OR(=($0, 1), =($0, 2), =($0, 
3)))])
LogicalJoin(condition=[=($0, $10)], joinType=[inner])
  LogicalJoin(condition=[=($0, $9)], joinType=[inner])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalAggregate(group=[{0}])
  LogicalProject(EMPNO=[$0])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
  LogicalAggregate(group=[{0}])
LogicalValues(tuples=[[{ 1 }, { 2 }, { 3 }, { 4 }, { 5 }, { 6 }, { 7 }, 
{ 8 }, { 9 }, { 10 }, { 11 }, { 12 }, { 13 }, { 14 }, { 15 }, { 16 }, { 17 }, { 
18 }, { 19 }, { 20 }, { 21 }, { 22 }, { 23 }, { 24 }, { 25 }]])
{noformat}
where filter has incorrect condition: {{OR(true, true)}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (CALCITE-2069) RexSimplify.removeNullabilityCast() always removes cast for operand with ANY type

2017-11-28 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2069:


 Summary: RexSimplify.removeNullabilityCast() always removes cast 
for operand with ANY type
 Key: CALCITE-2069
 URL: https://issues.apache.org/jira/browse/CALCITE-2069
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


When a field is received from Dynamic Table, its type left {{ANY}}, and it is 
used in the filter condition with the cast, which actually should produce 
physical cast (for example we are trying to cast varchar to boolean) 
{{RexSimplify.removeNullabilityCast()}} removes this cast and lefts only field 
in condition.
This test helps to observe this issue:
{code:java}
  @Test public void testFilterCastAny() {
final RelBuilder builder = RelBuilder.create(config().build());
final RelDataType intType = 
builder.getTypeFactory().createSqlType(SqlTypeName.ANY);
RelNode root =
  builder.scan("EMP")
.filter(
  builder.cast(
builder.patternField("varchar_field", intType, 0),
SqlTypeName.BOOLEAN))
.build();
assertThat(str(root),
  is("LogicalFilter(condition=[CAST(varchar_field.$0):BOOLEAN NOT NULL])\n"
  + "  LogicalTableScan(table=[[scott, EMP]])\n"));
  }
{code}

It happens because {{SqlTypeUtil.equalSansNullability()}} returns true if any 
of its arguments has {{ANY}} type.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (CALCITE-2066) RelOptUtil.splitJoinCondition() could not split condition with case after applying FilterReduceExpressionsRule

2017-11-27 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2066:


 Summary: RelOptUtil.splitJoinCondition() could not split condition 
with case after applying FilterReduceExpressionsRule
 Key: CALCITE-2066
 URL: https://issues.apache.org/jira/browse/CALCITE-2066
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


Query 
{code:sql}
select * from emp a emp b where coalesce(a.mgr, -1) = coalesce(b.mgr, -1)
{code}
has logical plan
{noformat}
LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], 
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], EMPNO0=[$9], ENAME0=[$10], 
JOB0=[$11], MGR0=[$12], HIREDATE0=[$13], SAL0=[$14], COMM0=[$15], 
DEPTNO0=[$16], SLACKER0=[$17])
  LogicalFilter(condition=[=(CASE(IS NOT NULL($3), $3, -1), CASE(IS NOT 
NULL($12), $12, -1))])
LogicalJoin(condition=[true], joinType=[inner])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{noformat}
After applying {{FilterReduceExpressionsRule}} condition in filter is changed to
{noformat}
CASE(IS NOT NULL($12), CASE(IS NOT NULL($3), =($3, $12), =(-1, $12)), CASE(IS 
NOT NULL($3), =($3, -1), CAST(true):BOOLEAN))
{noformat}

When this condition is passed into {{RelOptUtil.splitJoinCondition()}} method, 
{{leftJoinKeys}} and  {{rightJoinKeys}} lists are left empty and passed 
condition is returned.

*Note:* {{RelOptUtil.splitJoinCondition()}} works correctly for the initial 
condition without applying {{FilterReduceExpressionsRule}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (CALCITE-2055) Add check for year/month/day range

2017-11-15 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2055:


 Summary: Add check for year/month/day range
 Key: CALCITE-2055
 URL: https://issues.apache.org/jira/browse/CALCITE-2055
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


Currently, if the year that is passed into {{DateString}} constructor has five 
digits, the first digit is trimmed. This trimming happens in the 
{{DateTimeStringUtils.int4()}} method:
{code:sql}
  private static void int4(StringBuilder buf, int i) {
buf.append((char) ('0' + (i / 1000) % 10));
buf.append((char) ('0' + (i / 100) % 10));
buf.append((char) ('0' + (i / 10) % 10));
buf.append((char) ('0' + i % 10));
  }
{code}
The same problem with month and day values.

Instead of trimming the value, the correct behaviour is to throw an exception 
if any of the values are outside the expected range.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (CALCITE-2043) Use custom RelBuilder implementation in some rules

2017-11-08 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2043:


 Summary: Use custom RelBuilder implementation in some rules
 Key: CALCITE-2043
 URL: https://issues.apache.org/jira/browse/CALCITE-2043
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


Some rules do not have constructors with RelBuilderFactory. 

List of such rules that are used in Drill:
* AggregateRemoveRule
* ProjectRemoveRule
* SortRemoveRule
* ProjectWindowTransposeRule
* ExpandConversionRule
* ConverterRule
* ExpandConversionRule
* JoinToMultiJoinRule
* ProjectToWindowRule



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (CALCITE-2026) AssertionError when ProjectReduceExpressionsRule simplifies project expressions

2017-11-01 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2026:


 Summary: AssertionError when ProjectReduceExpressionsRule 
simplifies project expressions
 Key: CALCITE-2026
 URL: https://issues.apache.org/jira/browse/CALCITE-2026
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde
Priority: Major


When applying ProjectReduceExpressionsRule on project which expression has a 
non-nullable type, but after simplifying expression is null, the query fails 
with the assertion error:
{noformat}
java.lang.AssertionError: Type mismatch:
rel rowtype:
RecordType(INTEGER EXPR$0) NOT NULL
equivRel rowtype:
RecordType(INTEGER NOT NULL EXPR$0) NOT NULL
at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31) 
~[calcite-core-1.13.0.jar:1.13.0]
at org.apache.calcite.plan.RelOptUtil.equal(RelOptUtil.java:1868) 
~[calcite-core-1.13.0.jar:1.13.0]
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:855)
 ~[calcite-core-1.13.0.jar:1.13.0]
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:883)
 ~[calcite-core-1.13.0.jar:1.13.0]
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1767)
 ~[calcite-core-1.13.0.jar:1.13.0]
at 
org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:135)
 ~[calcite-core-1.13.0.jar:1.13.0]
at 
org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:234) 
~[calcite-core-1.13.0.jar:1.13.0]
at 
org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:268)
 ~[calcite-core-1.13.0.jar:1.13.0]
at 
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
 ~[calcite-core-1.13.0.jar:1.13.0]
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:650)
 ~[calcite-core-1.13.0.jar:1.13.0]
at 
org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:368) 
~[calcite-core-1.13.0.jar:1.13.0]
{noformat}

The similar bug was described in CALCITE-1502, since there was a problem with 
non-nullable and nullable types in different case branches, but this bug 
appears when project expression is transformed into the expression with 
nullable type.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (CALCITE-2018) Queries failed with AssertionError: rel has lower cost than best cost of subset

2017-10-19 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2018:


 Summary: Queries failed with AssertionError: rel has lower cost 
than best cost of subset
 Key: CALCITE-2018
 URL: https://issues.apache.org/jira/browse/CALCITE-2018
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.13.0
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


*Problem description*
When debug is enabled, unit tests 
* MaterializationTest.testMaterializationSubstitution2
* MaterializationTest.testJoinMaterializationUKFK8
* MaterializationTest.testJoinMaterializationUKFK6
* JdbcTest.testWhereNot

unit tests are failed with error AssertionError: rel has lower cost than best 
cost of subset.

Full stack trace for test 
{{MaterializationTest.testMaterializationSubstitution2}}:
{noformat}
java.lang.AssertionError: rel 
[rel#245:EnumerableUnion.ENUMERABLE.[](input#0=rel#246:Subset#5.ENUMERABLE.[],input#1=rel#239:Subset#6.ENUMERABLE.[0],all=true)]
 has lower cost {14.0 rows, 19.0 cpu, 0.0 io} than best cost {15.0 rows, 20.0 
cpu, 0.0 io} of subset [rel#243:Subset#7.ENUMERABLE.[]]

at 
org.apache.calcite.plan.volcano.VolcanoPlanner.validate(VolcanoPlanner.java:906)
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:866)
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:883)
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:101)
at 
org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:336)
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1495)
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:863)
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:883)
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1766)
at 
org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:135)
at 
org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:234)
at 
org.apache.calcite.rel.rules.FilterProjectTransposeRule.onMatch(FilterProjectTransposeRule.java:143)
at 
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:650)
at org.apache.calcite.tools.Programs$5.run(Programs.java:326)
at 
org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:387)
at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:187)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:318)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:229)
at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:786)
at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640)
at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610)
at org.apache.calcite.schema.Schemas.prepare(Schemas.java:346)
at 
org.apache.calcite.materialize.MaterializationService$DefaultTableFactory.createTable(MaterializationService.java:374)
at 
org.apache.calcite.materialize.MaterializationService.defineMaterialization(MaterializationService.java:137)
at 
org.apache.calcite.materialize.MaterializationService.defineMaterialization(MaterializationService.java:99)
at 
org.apache.calcite.schema.impl.MaterializedViewTable$MaterializedViewTableMacro.(MaterializedViewTable.java:110)
at 
org.apache.calcite.schema.impl.MaterializedViewTable$MaterializedViewTableMacro.(MaterializedViewTable.java:100)
at 
org.apache.calcite.schema.impl.MaterializedViewTable.create(MaterializedViewTable.java:81)
at org.apache.calcite.model.ModelHandler.visit(ModelHandler.java:364)
at 
org.apache.calcite.model.JsonMaterialization.accept(JsonMaterialization.java:42)
at org.apache.calcite.model.JsonSchema.visitChildren(JsonSchema.java:98)
at 
org.apache.calcite.model.JsonMapSchema.visitChildren(JsonMapSchema.java:48)
at 
org.apache.calcite.model.ModelHandler.populateSchema(ModelHandler.java:257)
at org.apache.calcite.model.ModelHandler.visit(ModelHandler.java:273)
at 
org.apache.calcite.model.JsonCustomSchema.accept(JsonCustomSchema.java:45)
at org.apache.calcite.model.ModelHandler.visit(ModelHandler.java:196)
at org.apache.calcite.model.ModelHandler.(ModelHandler.java:88)
at org.apache.calcite.jdbc.Driver$1.onConnectionInit(Driver.java:104)
at 
org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:139)
at 

[jira] [Created] (CALCITE-1944) Wrong plan for query with window functions and subquery with star

2017-08-15 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-1944:


 Summary: Wrong plan for query with window functions and subquery 
with star
 Key: CALCITE-1944
 URL: https://issues.apache.org/jira/browse/CALCITE-1944
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.13.0
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


*Problem description*
Calcite builds the wrong plan for the query with a window function and subquery 
with a star:
{code:sql}
SELECT SUM(n_nationkey) OVER w
FROM
  (SELECT *
   FROM SALES.NATION) subQry WINDOW w AS (PARTITION BY REGION
  ORDER BY n_nationkey)
{code}  
Plan:
{noformat}
LogicalProject(EXPR$0=[CASE(>(COUNT(ITEM($0, 'N_NATIONKEY')) OVER (PARTITION BY 
$0 ORDER BY $0 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW), 0), 
$SUM0(ITEM($0, 'N_NATIONKEY')) OVER (PARTITION BY $0 ORDER BY $0 RANGE BETWEEN 
UNBOUNDED PRECEDING AND CURRENT ROW), null)])
  LogicalProject(**=[$0])
LogicalTableScan(table=[[CATALOG, SALES, NATION]])
{noformat}
Columns in PARTITION BY and ORDER BY clauses are {{$0}} but they should be 
{{ITEM($0, 'REGION')}} and {{ITEM($0, 'N_NATIONKEY')}} respectively.
So correct plan should be
{noformat}
LogicalProject(EXPR$0=[CASE(>(COUNT(ITEM($0, 'N_NATIONKEY')) OVER (PARTITION BY 
ITEM($0, 'REGION') ORDER BY ITEM($0, 'N_NATIONKEY') RANGE BETWEEN UNBOUNDED 
PRECEDING AND CURRENT ROW), 0), $SUM0(ITEM($0, 'N_NATIONKEY')) OVER (PARTITION 
BY ITEM($0, 'REGION') ORDER BY ITEM($0, 'N_NATIONKEY') RANGE BETWEEN UNBOUNDED 
PRECEDING AND CURRENT ROW), null)])
  LogicalProject(**=[$0])
LogicalTableScan(table=[[CATALOG, SALES, NATION]])
{noformat}

*Root cause*
In CALCITE-1150 added dynamic star column and dynamic record type support but 
[SqlValidatorImpl.validateWindowClause()|https://github.com/apache/calcite/blob/01c5446138d419a85678bff7db06eabb4cd39846/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L3354]
 method does not call [expand(SqlNode expr, SqlValidatorScope 
scope)|https://github.com/apache/calcite/blob/01c5446138d419a85678bff7db06eabb4cd39846/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L4795]
 for columns in partition and order lists. 

Therefore when executes this line 
[windowList.validate(this, 
windowScope);|https://github.com/apache/calcite/blob/01c5446138d419a85678bff7db06eabb4cd39846/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L3392]
 
in the [validateIdentifier(SqlIdentifier id, SqlValidatorScope 
scope)|https://github.com/apache/calcite/blob/01c5446138d419a85678bff7db06eabb4cd39846/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L2778]
 method simple field name is replaced by the star field. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)