[jira] [Created] (CALCITE-6429) Arrow adapter should default to the Enumerable convention for unsupported features
Alessandro Solimando created CALCITE-6429:
-
Summary: Arrow adapter should default to the Enumerable convention
for unsupported features
Key: CALCITE-6429
URL: https://issues.apache.org/jira/browse/CALCITE-6429
Project: Calcite
Issue Type: Sub-task
Affects Versions: 1.37.0
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando
Fix For: 1.38.0
Currently, the adapter fails with different errors when one of the known
unsupported features is used, while it might simply catch the
"UnsupportedOperationException" and bail out, thus allowing to generate a plan
where the unsupported operation is implemented in the Enumerable convention.
For instance, currently several tests are disabled for this reason, like
[testArrowProjectFieldsWithDisjunctiveFilter|https://github.com/apache/calcite/blob/be9b860ebf3143dcbdbd0ee1777e604f0ace7b3c/arrow/src/test/java/org/apache/calcite/adapter/arrow/ArrowAdapterTest.java#L242],
because it would fail as follows:
{code:java}
Error while executing SQL "select "intField", "stringField"
from arrowdata
where "intField"=12 or "stringField"='12'": Error while applying rule
ArrowFilterRule, args
[rel#31:LogicalFilter.NONE.[](input=RelSubset#15,condition=OR(=($0, 12), =($1,
'12'))), rel#1:ArrowTableScan.ARROW.[](table=[ARROW, ARROWDATA],fields=[0, 1,
2, 3])]
java.sql.SQLException: Error while executing SQL "select "intField",
"stringField"
from arrowdata
where "intField"=12 or "stringField"='12'": Error while applying rule
ArrowFilterRule, args
[rel#31:LogicalFilter.NONE.[](input=RelSubset#15,condition=OR(=($0, 12), =($1,
'12'))), rel#1:ArrowTableScan.ARROW.[](table=[ARROW, ARROWDATA],fields=[0, 1,
2, 3])]
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:228)
at org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:566)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.lambda$returns$1(CalciteAssert.java:1495)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.withConnection(CalciteAssert.java:1434)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1493)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1483)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1446)
at
org.apache.calcite.adapter.arrow.ArrowAdapterTest.testArrowProjectFieldsWithDisjunctiveFilter(ArrowAdapterTest.java:260)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
at
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at
org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:45)
at
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
at
[jira] [Created] (CALCITE-6388) PsTableFunction throws NumberFormatException when the 'user' column has spaces
Alessandro Solimando created CALCITE-6388:
-
Summary: PsTableFunction throws NumberFormatException when the
'user' column has spaces
Key: CALCITE-6388
URL: https://issues.apache.org/jira/browse/CALCITE-6388
Project: Calcite
Issue Type: Bug
Components: os-adapter
Affects Versions: 1.36.0
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando
Fix For: 1.37.0
Line parsing splits on spaces
([PsTableFunction.java#L77|https://github.com/apache/calcite/blob/aa8d81bf1ff39e3632aeb856fc4cc247ce9727e5/plus/src/main/java/org/apache/calcite/adapter/os/PsTableFunction.java#L77]),
which breaks whenever the "user" contains at least a space.
An example output on my laptop is as follows:
{noformat}
$ ps ax -o
ppid=,pid=,pgid=,tpgid=,stat=,user=,pcpu=,pmem=,vsz=,rss=,tty=,start=,time=,uid=,ruid=,sess=,comm=
| grep startup
1 6728 6728 0 S startup user 0.0 0.0 410266096 2528 ??
11Apr24 0:16.97 501 501 0 /usr/sbin/distnoted
1 6729 6729 0 SN startup user 0.0 0.1 410332256 17616 ??
11Apr24 0:42.41 501 501 0
/System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework/Versions/A/Support/mdbulkimport
1 6750 6750 0 S startup user 0.0 0.0 410376144 13344 ??
11Apr24 0:40.39 501 501 0 /usr/libexec/lsd
1 10148 10148 0 S startup user 0.0 0.0 410354816 5488 ??
8:26PM 0:00.31 501 501 0 /usr/libexec/containermanagerd
1 95313 95313 0 S startup user 0.0 0.0 410357344 6576 ??
Fri05PM 0:00.32 501 501 0 /usr/libexec/trustd{noformat}
When running the test it fails with the following stack trace:
{code:java}
Error while executing SQL "select distinct `user` from ps": while parsing value
[user] of field [pcpu] in line [ 1 6728 6728 0 S startup user
0.0 0.0 410266096 2528 ?? 11Apr24 0:16.95 501 501 0
/usr/sbin/distnoted]
java.sql.SQLException: Error while executing SQL "select distinct `user` from
ps": while parsing value [user] of field [pcpu] in line [ 1 6728 6728 0
S startup user 0.0 0.0 410266096 2528 ?? 11Apr24 0:16.95
501 501 0 /usr/sbin/distnoted]
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:228)
at org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:566)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.lambda$returns$1(CalciteAssert.java:1495)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.withConnection(CalciteAssert.java:1434)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1493)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1483)
at
org.apache.calcite.adapter.os.OsAdapterTest.testPsDistinct(OsAdapterTest.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
at
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at
org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:45)
at
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at
[jira] [Created] (CALCITE-6307) Sonar analysis in CI fails with 'java.lang.OutOfMemoryError: Java heap space'
Alessandro Solimando created CALCITE-6307:
-
Summary: Sonar analysis in CI fails with
'java.lang.OutOfMemoryError: Java heap space'
Key: CALCITE-6307
URL: https://issues.apache.org/jira/browse/CALCITE-6307
Project: Calcite
Issue Type: Bug
Components: tests
Affects Versions: 1.36.0
Reporter: Alessandro Solimando
Recently there has been several Sonar failures due to OOM, one example is the
following:
[https://ci-builds.apache.org/blue/organizations/jenkins/Calcite%2FCalcite-sonar/detail/PR-3687/9/pipeline]
In what follows a relevant portion of the execution logs:
{code:java}
Expiring Daemon because JVM heap space is exhausted
> Task :sonar FAILED
Build calcite FAILURE reason:
Execution failed for task ':sonar':
java.lang.OutOfMemoryError: Java heap space
at B.A.A.A.A.D.B(na:26)
at B.A.A.A.A.D.A(na:163)
at B.A.A.A.A.D.B(na:1209)
at B.A.A.A.A.D.B(na:2138)
at B.A.A.A.A.D.B(na:2138)
at B.A.A.A.A.D.B(na:2138)
at B.A.A.A.A.D.B(na:2138)
at B.A.A.A.A.D.B(na:2138)
at B.A.A.A.A.D.B(na:2221)
at B.A.A.A.A.D$_C.A(na:2102)
at com.sonarsource.A.A.U.A(na:3338)
at com.sonarsource.A.A.B.F.A(na:501)
at com.sonarsource.A.A.U.E(na:1630)
at com.sonarsource.A.A.Z.A(na:2867)
at com.sonarsource.A.A.Z.A(na:2409)
at com.sonarsource.A.A.Z.A(na:2223)
at com.sonarsource.A.A.Z.A(na:920)
at com.sonarsource.A.A.Z.E(na:1730)
at com.sonarsource.A.A.Z.A(na:242)
at com.sonarsource.A.A.Z.A(na:671)
at com.sonarsource.A.A.Z.A(na:2244)
at com.sonarsource.A.F.executeChecksOnFunction(na:896)
at com.sonarsource.A.F.executeChecks(na:1668)
at com.sonarsource.A.F.executeSensor(na:1339)
at com.sonarsource.A.F.execute(na:2143)
at
org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:62)
at
org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:75)
at
org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:51)
at
org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:64)
at
org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
at
org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
at
org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:192)
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sonar'.
> Java heap space {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6243) Upgrade Cassandra to 4.1.3 and DataStax driver for Cassandra to 4.17.0
Alessandro Solimando created CALCITE-6243: - Summary: Upgrade Cassandra to 4.1.3 and DataStax driver for Cassandra to 4.17.0 Key: CALCITE-6243 URL: https://issues.apache.org/jira/browse/CALCITE-6243 Project: Calcite Issue Type: Task Components: cassandra-adapter Affects Versions: 1.36.0 Reporter: Alessandro Solimando Assignee: Alessandro Solimando Fix For: 1.37.0 Upgrading the following dependencies: * the Cassandra version from 4.0.1 to 4.1.3 * the DataStax Apache Cassandra driver from 4.13.0 to 4.17.0 >From 4.1.x Windows support is dropped by Cassandra (CASSANDRA-16956), so >Cassandra tests are now skipped when executing on Windows. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5505) JavaCC warns about missing LOOKAHEAD directives in Parser.jj
Alessandro Solimando created CALCITE-5505:
-
Summary: JavaCC warns about missing LOOKAHEAD directives in
Parser.jj
Key: CALCITE-5505
URL: https://issues.apache.org/jira/browse/CALCITE-5505
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.32.0
Reporter: Alessandro Solimando
JavaCC is reporting several warnings for insufficient lookahead for Parser.jj,
an example of log displaying the warnings is here:
[https://ci-builds.apache.org/job/Calcite/job/Calcite-sonar/job/main/18/consoleFull]
Here is the relevant extract from the aforementioned log file:
{noformat}
> Task :core:javaCCMain
Java Compiler Compiler Version 4.0 (Parser Generator)
(type "javacc" with no arguments for help)
Reading from file
/home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/fmpp/fmppMain/javacc/Parser.jj
. . .
Warning: Output directory
"/home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/javacc/javaCCMain/org/apache/calcite/sql/parser/impl"
does not exist. Creating the directory.
Note: UNICODE_INPUT option is specified. Please make sure you create the
parser/lexer using a Reader with the correct character encoding.
Warning: Choice conflict involving two expansions at
line 4930, column 5 and line 4956, column 5 respectively.
A common prefix is: "MICROSECOND"
Consider using a lookahead of 2 for earlier expansion.
Warning: Choice conflict involving two expansions at
line 4931, column 5 and line 4956, column 5 respectively.
A common prefix is: "MILLISECOND"
Consider using a lookahead of 2 for earlier expansion.
Warning: Choice conflict involving two expansions at
line 4936, column 5 and line 4956, column 5 respectively.
A common prefix is: "DOW"
Consider using a lookahead of 2 for earlier expansion.
Warning: Choice conflict involving two expansions at
line 4937, column 5 and line 4956, column 5 respectively.
A common prefix is: "DOY"
Consider using a lookahead of 2 for earlier expansion.
Warning: Choice conflict involving two expansions at
line 4938, column 5 and line 4956, column 5 respectively.
A common prefix is: "ISODOW"
Consider using a lookahead of 2 for earlier expansion.
Warning: Choice conflict involving two expansions at
line 4939, column 5 and line 4956, column 5 respectively.
A common prefix is: "ISOYEAR"
Consider using a lookahead of 2 for earlier expansion.
Warning: Choice conflict involving two expansions at
line 4940, column 5 and line 4956, column 5 respectively.
A common prefix is: "WEEK"
Consider using a lookahead of 2 for earlier expansion.
Warning: Choice conflict involving two expansions at
line 4950, column 5 and line 4956, column 5 respectively.
A common prefix is: "QUARTER"
Consider using a lookahead of 2 for earlier expansion.
Warning: Choice conflict involving two expansions at
line 4952, column 5 and line 4956, column 5 respectively.
A common prefix is: "EPOCH"
Consider using a lookahead of 2 for earlier expansion.
Warning: Choice conflict involving two expansions at
line 4953, column 5 and line 4956, column 5 respectively.
A common prefix is: "DECADE"
Consider using a lookahead of 2 for earlier expansion.
Warning: Choice conflict involving two expansions at
line 4954, column 5 and line 4956, column 5 respectively.
A common prefix is: "CENTURY"
Consider using a lookahead of 2 for earlier expansion.
Warning: Choice conflict involving two expansions at
line 4955, column 5 and line 4956, column 5 respectively.
A common prefix is: "MILLENNIUM"
Consider using a lookahead of 2 for earlier expansion.
Warning: Choice conflict involving two expansions at
line 6549, column 9 and line 6551, column 9 respectively.
A common prefix is: "WEEK" "("
Consider using a lookahead of 3 or more for earlier expansion.
File "TokenMgrError.java" does not exist. Will create one.
File "ParseException.java" does not exist. Will create one.
File "Token.java" does not exist. Will create one.
File "SimpleCharStream.java" does not exist. Will create one.
Parser generated with 0 errors and 14 warnings.{noformat}
We are probably missing one or two LOOKAHEAD directives in the parser.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5345) UnionPullUpConstantsRule could also pull up constants requiring a cast
Alessandro Solimando created CALCITE-5345:
-
Summary: UnionPullUpConstantsRule could also pull up constants
requiring a cast
Key: CALCITE-5345
URL: https://issues.apache.org/jira/browse/CALCITE-5345
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.32.0
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando
Consider the following SQL query:
{code:java}
select deptno, ename from emp where deptno = 1.0
union all
select deptno, ename from emp where deptno = 1.0
{code}
The associated plan is as follows:
{code:java}
LogicalUnion(all=[true])
LogicalProject(DEPTNO=[$1], ENAME=[$0])
LogicalFilter(condition=[=(CAST($1):DECIMAL(11, 1) NOT NULL, 1.0)])
LogicalProject(ENAME=[$1], DEPTNO=[$7])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalProject(DEPTNO=[$1], ENAME=[$0])
LogicalFilter(condition=[=(CAST($1):DECIMAL(11, 1) NOT NULL, 1.0)])
LogicalProject(ENAME=[$1], DEPTNO=[$7])
LogicalTableScan(table=[[CATALOG, SALES, EMP]]){code}
Note that since _deptno_ is of type {_}int{_}, a cast is needed in the filter
({_}i.e., LogicalFilter(condition=[=(CAST($1):DECIMAL(11, 1) NOT NULL,
1.0)]){_}).
{_}UnionPullUpConstantsRule{_}, as currently written, processes only
(pulled-up) predicates of the form "{_}=($i, $literal){_}", while now that
CALCITE-5337 is present, it could also process "{_}=(CAST($i, $type),
$literal){_}", because the need of a cast is recognized and the cast added in
the projection when the constant is pulled up.
The aforementioned query would be optimized in this way:
{code:java}
LogicalProject(DEPTNO=[1], ENAME=[$0])
LogicalUnion(all=[true])
LogicalProject(ENAME=[$0])
LogicalFilter(condition=[=(CAST($1):DECIMAL(11, 1) NOT NULL, 1.0)])
LogicalProject(ENAME=[$1], DEPTNO=[$7])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalProject(ENAME=[$0])
LogicalFilter(condition=[=(CAST($1):DECIMAL(11, 1) NOT NULL, 1.0)])
LogicalProject(ENAME=[$1], DEPTNO=[$7])
LogicalTableScan(table=[[CATALOG, SALES, EMP]]){code}
Without this improvement, the plan would not change after applying
{_}UnionPullUpConstantsRule{_}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5340) Tests should fail when actual and expected XML files are not identical
Alessandro Solimando created CALCITE-5340: - Summary: Tests should fail when actual and expected XML files are not identical Key: CALCITE-5340 URL: https://issues.apache.org/jira/browse/CALCITE-5340 Project: Calcite Issue Type: Bug Components: build, tests Affects Versions: 1.32.0 Reporter: Alessandro Solimando Calcite has several XML reference files for tests. When a new test is added, those files should be updated by copying the new, automatically-generated, version. Sometimes the files are updated manually, and differ from the automatic generation, which makes it harder for the next developer to rely on automatic generation. Following the [discussion|https://lists.apache.org/thread/m567vhcmxyt7thprbs22botpnvjg98mj] in the ML, it is desirable to have tests failing upon discrepancies. This ticket aims at tracking the effort towards this goal. _DiffRepository_ class is responsible for diffing, so it's the first candidate to consider for adding such a check. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5337) UnionPullUpConstantsRule produces an invalid plan when pulling up constants for nullable fields
Alessandro Solimando created CALCITE-5337:
-
Summary: UnionPullUpConstantsRule produces an invalid plan when
pulling up constants for nullable fields
Key: CALCITE-5337
URL: https://issues.apache.org/jira/browse/CALCITE-5337
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.32.0
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando
Fix For: 1.33.0
The rule pulls up constants without checking/adjusting nullability to match
that of the field type.
Here is the stack-trace when a nullable type is involved:
{code:java}
java.lang.AssertionError: Cannot add expression of different type to set:
set type is RecordType(INTEGER DEPTNO, VARCHAR(20) ENAME) NOT NULL
expression type is RecordType(INTEGER NOT NULL DEPTNO, VARCHAR(20) ENAME) NOT
NULL
set is
rel#37:LogicalUnion.(input#0=HepRelVertex#33,input#1=HepRelVertex#33,all=true)
expression is LogicalProject(DEPTNO=[1], ENAME=[$0])
LogicalUnion(all=[true])
LogicalProject(ENAME=[$1])
LogicalProject(DEPTNO=[$1], ENAME=[$0])
LogicalFilter(condition=[=($1, 1)])
LogicalProject(ENAME=[$1], DEPTNO=[$7])
LogicalTableScan(table=[[CATALOG, SALES, EMPNULLABLES]])
LogicalProject(ENAME=[$1])
LogicalProject(DEPTNO=[$1], ENAME=[$0])
LogicalFilter(condition=[=($1, 1)])
LogicalProject(ENAME=[$1], DEPTNO=[$7])
LogicalTableScan(table=[[CATALOG, SALES, EMPNULLABLES]]) at
org.apache.calcite.plan.RelOptUtil.verifyTypeEquivalence(RelOptUtil.java:391)
at org.apache.calcite.plan.hep.HepRuleCall.transformTo(HepRuleCall.java:60)
at
org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:269)
at
org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:284)
at
org.apache.calcite.rel.rules.UnionPullUpConstantsRule.onMatch(UnionPullUpConstantsRule.java:138)
at
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:337)
at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:556)
at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:420)
at
org.apache.calcite.plan.hep.HepPlanner.executeRuleInstance(HepPlanner.java:243)
at
org.apache.calcite.plan.hep.HepInstruction$RuleInstance$State.execute(HepInstruction.java:178)
at
org.apache.calcite.plan.hep.HepPlanner.lambda$executeProgram$0(HepPlanner.java:211)
at com.google.common.collect.ImmutableList.forEach(ImmutableList.java:422)
at
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:210)
at org.apache.calcite.plan.hep.HepProgram$State.execute(HepProgram.java:118)
at
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:205)
at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:191)
at
org.apache.calcite.test.RelOptFixture.checkPlanning(RelOptFixture.java:378)
at org.apache.calcite.test.RelOptFixture.check(RelOptFixture.java:330)
at org.apache.calcite.test.RelOptFixture.check(RelOptFixture.java:314)
at
org.apache.calcite.test.RelOptRulesTest.testPullConstantThroughUnion(RelOptRulesTest.java:3745)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
at
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at
org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:46)
at
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at
org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at
org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at
[jira] [Created] (CALCITE-5306) Support LTS JDKs and latest in testing
Alessandro Solimando created CALCITE-5306: - Summary: Support LTS JDKs and latest in testing Key: CALCITE-5306 URL: https://issues.apache.org/jira/browse/CALCITE-5306 Project: Calcite Issue Type: Test Components: tests Affects Versions: 1.32.0 Reporter: Alessandro Solimando Assignee: Alessandro Solimando Following [this discussion|https://lists.apache.org/thread/txh3n7r1sygqs69084z9m7g6r1hjbmn8] in the ML, we agreed on supporting only LTS JDK versions plus latest in our test environment (Github actions, AppVeyor and Travis). JDK 8 for the usual reasons is an exception to the rule and will be kept. All non-LTS EOL versions will be upgraded to the next LTS version (_e.g._, JDK 15 will be replaced by JDK 17). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5011) CassandraAdapterDataTypesTest fails with initialization error
Alessandro Solimando created CALCITE-5011:
-
Summary: CassandraAdapterDataTypesTest fails with initialization
error
Key: CALCITE-5011
URL: https://issues.apache.org/jira/browse/CALCITE-5011
Project: Calcite
Issue Type: Bug
Components: tests
Affects Versions: 1.29.0
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando
{noformat}
> Task :cassandra:test FAILED
FAILURE 0.0sec, org.apache.calcite.test.CassandraAdapterTest >
initializationError
com.datastax.oss.driver.api.core.servererrors.WriteTimeoutException:
Cassandra timeout during SIMPLE write query at consistency LOCAL_ONE (1 replica
were required but only 0 acknowledged the write)
at
com.datastax.oss.driver.api.core.servererrors.WriteTimeoutException.copy(WriteTimeoutException.java:96)
at
com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149)
at
com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:53)
at
com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:30)
at
com.datastax.oss.driver.internal.core.session.DefaultSession.execute(DefaultSession.java:230)
at
com.datastax.oss.driver.api.core.cql.SyncCqlSession.execute(SyncCqlSession.java:54)
at
com.datastax.oss.driver.api.core.cql.SyncCqlSession.execute(SyncCqlSession.java:78)
at
org.cassandraunit.utils.CqlOperations.lambda$execute$0(CqlOperations.java:18)
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
at org.cassandraunit.CQLDataLoader.load(CQLDataLoader.java:34)
at
org.apache.calcite.test.CassandraAdapterTest.load(CassandraAdapterTest.java:52)
FAILURE 38.2sec,1 completed, 1 failed, 0 skipped,
org.apache.calcite.test.CassandraAdapterTest
{noformat}
The root cause is probably a session leak, as suggested by the warning:
{noformat}
CassandraAdapterDataTypesTest > testSimpleTypesRowType() STANDARD_OUT
2022-02-14 04:36:56,298 [ForkJoinPool-1-worker-1] WARN - You have too many
session instances: 15 active, expected less than 4 (see
'advanced.session-leak.threshold' in the configuration)
> Task :core:forbiddenApisMain
{noformat}
For the full error see:
https://ci.appveyor.com/project/ApacheSoftwareFoundation/calcite/builds/42562506/job/4ppus57wtv36689d
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (CALCITE-4991) Improve RuleLogger to also print input rels in FULL_PLAN mode
Alessandro Solimando created CALCITE-4991:
-
Summary: Improve RuleLogger to also print input rels in FULL_PLAN
mode
Key: CALCITE-4991
URL: https://issues.apache.org/jira/browse/CALCITE-4991
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.29.0
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando
After using for some time now the new plan logging capabilities introduced via
_RuleEventLogger_ in CALCITE-4704, only printing the new rel produced by a rule
is not enough for complex cases, because the input rel(s) used by the rule are
not printed, and sometimes they are nowhere in the logs, making it hard to
understand what situation caused the appearance of a given plan of interest.
The present ticket aims at additionally printing, for _FULL_PLAN_ marker, the
input rels used by a given rule, before printing the output rel produced by the
rule application.
The output logs would evolve from:
{noformat}
022-01-21T02:41:27,466 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main]
calcite.RuleEventLogger: call#5: Apply rule
[HiveProjectFilterPullUpConstantsRule] to [rel#45:HiveProject,rel#56:HiveFilter]
2022-01-21T02:41:27,468 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main]
calcite.RuleEventLogger: call#5: Rule [HiveProjectFilterPullUpConstantsRule]
produced [rel#58:HiveProject]
2022-01-21T02:41:27,468 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main]
calcite.RuleEventLogger: call#5: Full plan for [rel#58:HiveProject]:
HiveProject(month=[CAST(202110):INTEGER])
HiveFilter(condition=[=($0, 202110)])
HiveTableScan(table=[[default, test2]], table:alias=[test2])
{noformat}
to:
{noformat}
022-01-21T02:41:27,466 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main]
calcite.RuleEventLogger: call#5: Apply rule
[HiveProjectFilterPullUpConstantsRule] to [rel#45:HiveProject,rel#56:HiveFilter]
2022-01-21T02:41:27,468 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main]
calcite.RuleEventLogger: call#5: Full plan for rule input [rel#45:HiveProject]:
HiveProject(month=[$0])
HiveFilter(condition=[=($0, 202110)])
HiveTableScan(table=[[default, test2]], table:alias=[test2])
2022-01-21T02:41:27,468 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main]
calcite.RuleEventLogger: call#5: Full plan for rule input [rel#56:HiveFilter]:
HiveFilter(condition=[=($0, 202110)])
HiveTableScan(table=[[default, test2]], table:alias=[test2])
2022-01-21T02:41:27,468 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main]
calcite.RuleEventLogger: call#5: Rule [HiveProjectFilterPullUpConstantsRule]
produced [rel#58:HiveProject]
2022-01-21T02:41:27,468 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main]
calcite.RuleEventLogger: call#5: Full plan for [rel#58:HiveProject]:
HiveProject(month=[CAST(202110):INTEGER])
HiveFilter(condition=[=($0, 202110)])
HiveTableScan(table=[[default, test2]], table:alias=[test2])
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (CALCITE-4917) Add test for 'IS NOT NULL(a) AND a=b' simplifies to "a=b" for UnknownAsFalse semantics
Alessandro Solimando created CALCITE-4917:
-
Summary: Add test for 'IS NOT NULL(a) AND a=b' simplifies to "a=b"
for UnknownAsFalse semantics
Key: CALCITE-4917
URL: https://issues.apache.org/jira/browse/CALCITE-4917
Project: Calcite
Issue Type: Test
Components: core
Affects Versions: 1.28.0
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando
Simplification (from _RexSimplify_ class) is mostly covered is
{_}RexProgramTest{_}, a test for
expressions of the form "IS NOT NULL(a) AND a=b" => "a=b" under the
"UnknownAsFalse" semantics and not simplified under "UnknownAsUnknown" one,
seems uncovered.
Since I had to write a test to make sure the simplification was as expected, I
assume others might end up doing the same, and that the test will both act as
documentation and it will also protect against regressions.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (CALCITE-4894) MV rewriting fails for expressions with more than a field reference in the view/query projection
Alessandro Solimando created CALCITE-4894:
-
Summary: MV rewriting fails for expressions with more than a field
reference in the view/query projection
Key: CALCITE-4894
URL: https://issues.apache.org/jira/browse/CALCITE-4894
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.28.0
Reporter: Alessandro Solimando
MV rewrite fails when at least one expression in the project of either the view
or the query references, directly or indirectly, to more than one field.
Consider a view with an expression of the form "f between 1 and 3" expression
(which under the hood becomes "f >= 1 and f <= 3", so effectively referencing
the same field twice):
{code:java}
@Test void testViewProjectWithBetween() {
sql("select s.\"time_id\", s.\"time_id\" between 1 and 3"
+ " from \"foodmart\".\"sales_fact_1997\" as s"
+ " where s.\"store_id\" = 1",
"select s.\"time_id\""
+ " from \"foodmart\".\"sales_fact_1997\" as s"
+ " where s.\"store_id\" = 1")
.withDefaultSchemaSpec(CalciteAssert.SchemaSpec.JDBC_FOODMART)
.ok();
}{code}
It fails as follows:
{noformat}
FAILURE 6.9sec, org.apache.calcite.test.MaterializedViewRelOptRulesTest >
testViewProjectWithBetween()
java.lang.AssertionError
at
org.apache.calcite.rel.rules.materialize.MaterializedViewRule.generateSwapTableColumnReferencesLineage(MaterializedViewRule.java:1046)
at
org.apache.calcite.rel.rules.materialize.MaterializedViewRule.rewriteExpressions(MaterializedViewRule.java:1005)
at
org.apache.calcite.rel.rules.materialize.MaterializedViewJoinRule.rewriteView(MaterializedViewJoinRule.java:278)
at
org.apache.calcite.rel.rules.materialize.MaterializedViewRule.perform(MaterializedViewRule.java:475)
at
org.apache.calcite.rel.rules.materialize.MaterializedViewOnlyFilterRule.onMatch(MaterializedViewOnlyFilterRule.java:50)
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:239)
at
org.apache.calcite.plan.volcano.IterativeRuleDriver.drive(IterativeRuleDriver.java:61)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:523)
at
org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:276)
at
org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:336)
at
org.apache.calcite.test.MaterializedViewRelOptRulesTest.optimize(MaterializedViewRelOptRulesTest.java:1210)
at
org.apache.calcite.test.AbstractMaterializedViewTest.checkMaterialize(AbstractMaterializedViewTest.java:109)
at
org.apache.calcite.test.AbstractMaterializedViewTest.access$000(AbstractMaterializedViewTest.java:69)
at
org.apache.calcite.test.AbstractMaterializedViewTest$Sql.ok(AbstractMaterializedViewTest.java:230)
at
org.apache.calcite.test.MaterializedViewRelOptRulesTest.testViewProjectWithBetween(MaterializedViewRelOptRulesTest.java:60){noformat}
Similarly when the same kind of expression is present in the query:
{code:java}
@Test void testQueryProjectWithBetween() {
sql("select *"
+ " from \"foodmart\".\"sales_fact_1997\" as s"
+ " where s.\"store_id\" = 1",
"select s.\"time_id\" between 1 and 3"
+ " from \"foodmart\".\"sales_fact_1997\" as s"
+ " where s.\"store_id\" = 1")
.withDefaultSchemaSpec(CalciteAssert.SchemaSpec.JDBC_FOODMART)
.ok();
} {code}
Code fails as follows:
{noformat}
FAILURE 5.5sec, org.apache.calcite.test.MaterializedViewRelOptRulesTest >
testQueryProjectWithBetween()
java.lang.AssertionError
at
org.apache.calcite.rel.rules.materialize.MaterializedViewJoinRule.rewriteView(MaterializedViewJoinRule.java:268)
at
org.apache.calcite.rel.rules.materialize.MaterializedViewRule.perform(MaterializedViewRule.java:475)
at
org.apache.calcite.rel.rules.materialize.MaterializedViewProjectFilterRule.onMatch(MaterializedViewProjectFilterRule.java:53)
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:239)
at
org.apache.calcite.plan.volcano.IterativeRuleDriver.drive(IterativeRuleDriver.java:61)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:523)
at
org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:276)
at
org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:336)
at
org.apache.calcite.test.MaterializedViewRelOptRulesTest.optimize(MaterializedViewRelOptRulesTest.java:1210)
at
org.apache.calcite.test.AbstractMaterializedViewTest.checkMaterialize(AbstractMaterializedViewTest.java:109)
at
org.apache.calcite.test.AbstractMaterializedViewTest.access$000(AbstractMaterializedViewTest.java:69)
at
[jira] [Created] (CALCITE-4816) Make Gradle pass the 'user.timezone' property to the test JVM (Avatica)
Alessandro Solimando created CALCITE-4816: - Summary: Make Gradle pass the 'user.timezone' property to the test JVM (Avatica) Key: CALCITE-4816 URL: https://issues.apache.org/jira/browse/CALCITE-4816 Project: Calcite Issue Type: Improvement Reporter: Alessandro Solimando Assignee: Alessandro Solimando Tests run in the forked JVM, only the explicit set of properties is passed there, which are listed [here|https://github.com/apache/calcite/blob/4b1e9ed1a513eae0c873a660b755bb4f27b39da9/build.gradle.kts#L711-L726] Add "user.timezone" to the list of properties to be passed along. The same change was applied to Calcite in order to facilitate local testing with different timezones. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4796) Travis links in README.md should point to 'travis-ci.com' instead of ' travis-ci.org'
Alessandro Solimando created CALCITE-4796:
-
Summary: Travis links in README.md should point to 'travis-ci.com'
instead of ' travis-ci.org'
Key: CALCITE-4796
URL: https://issues.apache.org/jira/browse/CALCITE-4796
Project: Calcite
Issue Type: Bug
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando
The travis links are broken, they point to 'travis-ci.org' instead of
'travis-ci.com', which leads to a broken landing page:
[https://travis-ci.org/github/apache/calcite]
{noformat}
Since June 15th, 2021, the building on travis-ci.org is ceased. Please use
travis-ci.com from now on. {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4793) CassandraAdapterDataTypesTest.testCollectionsInnerValues fails for guava <= 25.0-jre
Alessandro Solimando created CALCITE-4793:
-
Summary: CassandraAdapterDataTypesTest.testCollectionsInnerValues
fails for guava <= 25.0-jre
Key: CALCITE-4793
URL: https://issues.apache.org/jira/browse/CALCITE-4793
Project: Calcite
Issue Type: Bug
Components: cassandra-adapter
Affects Versions: 1.27.0
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando
Depending on the user timezone, the test fails because the value of the
timestamp field is not the expected one.
For instance, the following command:
{noformat}
./gradlew :cassandra:test --tests
"org.apache.calcite.test.CassandraAdapterDataTypesTest.testCollectionsInnerValues"
-Pguava.version=25.0-jre -Duser.timezone=GMT{noformat}
causes the following test failure:
{noformat}
java.lang.AssertionError: Expected: is "EXPR$0=1; EXPR$1=v1; 1=30;
2=30ff87; 3=2015-05-03 11:30:54\n" but: was "EXPR$0=1; EXPR$1=v1; 1=30;
2=30ff87; 3=2015-05-03 13:30:54\n"{noformat}
The issue is not present for guava versions >= 26.0-jre.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4790) Make Gradle pass the 'user.timezone' property to the test JVM
Alessandro Solimando created CALCITE-4790: - Summary: Make Gradle pass the 'user.timezone' property to the test JVM Key: CALCITE-4790 URL: https://issues.apache.org/jira/browse/CALCITE-4790 Project: Calcite Issue Type: Improvement Affects Versions: 1.27.0 Reporter: Alessandro Solimando Assignee: Alessandro Solimando Tests run in the forked JVM, only the explicit set of properties is passed there, which are listed [here|https://github.com/apache/calcite/blob/4b1e9ed1a513eae0c873a660b755bb4f27b39da9/build.gradle.kts#L711-L726] Add -Duser.timezone to the list of properties to be passed along. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4556) CalciteMetaImpl#createEmptyResultSet override of (Avatica) MetaImpl#createEmptyResultSet should not pass a class instance to CursorFactory#deduce
Alessandro Solimando created CALCITE-4556: - Summary: CalciteMetaImpl#createEmptyResultSet override of (Avatica) MetaImpl#createEmptyResultSet should not pass a class instance to CursorFactory#deduce Key: CALCITE-4556 URL: https://issues.apache.org/jira/browse/CALCITE-4556 Project: Calcite Issue Type: Bug Components: jdbc-driver Affects Versions: 1.26.0 Reporter: Alessandro Solimando Assignee: Alessandro Solimando Fix For: 1.27.0 In Avatica 1.18.0, `CursorFactory#deduce(List columns, Class resultClazz)` introduces a validation step requiring the names appearing in the column metadata to match fields of `resultClazz`, whenever the class is not null. `CalciteMetaImpl#createEmptyResultSet` overrides `MetaImpl#createEmptyResultSet` (class from Avatica), only to pass a value non null value to the `resultClazz` argument. Column metadata column names for Calcite internal metadata classes (_e.g._, `MetaColumn`, `MetaCatalog`, `MetaSchema`, _etc_.) are built using `MetaImpl#fieldMetadata` (transforming the name from camel case into upper-snake case), violating the new contract of `deduce`. This is not problematic, because `deduce` is only used to create an empty result set, we are only interested in preserving the sought column names, which is already the case for `MetaImpl#createEmptyResultSet`. `CalciteMetaImpl#createEmptyResultSet`, if adapted to match the changes coming from Avatica 1.18.0 would become identical to the `MetaImpl#createEmptyResultSet` it overrides, and it should therefore be removed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4503) Order of fields in records should follow that of the SQL types
Alessandro Solimando created CALCITE-4503:
-
Summary: Order of fields in records should follow that of the SQL
types
Key: CALCITE-4503
URL: https://issues.apache.org/jira/browse/CALCITE-4503
Project: Calcite
Issue Type: Bug
Components: avatica
Affects Versions: 1.17.0
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando
Fix For: 1.18.0
When dealing with records coming from Java classes, Avatica relies on the order
of fields coming from {{java.lang.Class#getFields}} instead of using the order
defined in the underlying SQL data type:
# [org.apache.calcite.avatica.MetaImpl#createGetter(int
ordinal)|https://github.com/apache/calcite-avatica/blob/ba20936bb1387793f34ae489760ec0cdbe205e4e/core/src/main/java/org/apache/calcite/avatica/MetaImpl.java#L145]
#
[org.apache.calcite.avatica.util.RecordIteratorCursor#RecordIteratorCursor(Iterator
iterator, Class
clazz)|https://github.com/apache/calcite-avatica/blob/ba20936bb1387793f34ae489760ec0cdbe205e4e/core/src/main/java/org/apache/calcite/avatica/util/RecordIteratorCursor.java#L42]
This behaviour prevents the change of fields orders, and it's particularly
problematic because {{#getFields}} is JVM-specific.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4436) Use the fields order from the struct type for 'ITEM(STRUCT, INDEX)' access
Alessandro Solimando created CALCITE-4436: - Summary: Use the fields order from the struct type for 'ITEM(STRUCT, INDEX)' access Key: CALCITE-4436 URL: https://issues.apache.org/jira/browse/CALCITE-4436 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.27.0 Reporter: Alessandro Solimando Assignee: Alessandro Solimando The index-based access via the "ITEM" operator for operands of row/struct type is currently experimental and disabled by default. Its implementation uses the fields order from `getDeclaredFields()`, a JVM-specific feature, this might lead to silent data corruption (see [PR-2230|https://github.com/apache/calcite/pull/2230]). The aim of the ticket is to improve the feature by relying on the safer fields order from the struct field type itself (_i.e._, `relType.getFields()[index]`) so that we can enable it by default. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4354) ITEM operator does not support synthetic struct type
Alessandro Solimando created CALCITE-4354: - Summary: ITEM operator does not support synthetic struct type Key: CALCITE-4354 URL: https://issues.apache.org/jira/browse/CALCITE-4354 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.27.0 Reporter: Alessandro Solimando The current implementation of the "ITEM" operator only supports struct/row type when this type if based on a class with named fields (that is, a non-synthetic type). For those non-synthetic struct types, the "ITEM" operator can be used to access named fields via a "string"-typed argument ([SqlItemOperator#96|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlItemOperator.java#L96]). The type checker accepts to apply "ITEM" on "ANY" type ([SqlItemOperator#49|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlItemOperator.java#L49]), and the operand checker only accepts "string"-based values ([SqlItemOperator.java#105|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlItemOperator.java#L105]). [SqlValidatorTest.java#L11933|https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/SqlValidatorTest.java#L11933] is an example of application of the "ITEM" operator over a struct with named fields. However, the “getAllowedSignatures” method ([SqlItemOperator.java#116|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlItemOperator.java#L116]) does not reflect this, and asserts a signature as follows: "[]" or "[]". "ITEM" operator could be enriched with the support of synthetic struct/row type, with a positional access to the fields with a signature "[]". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4301) Unit test 'testCollectionsInnerValues()' for Cassandra adapter is wrong
Alessandro Solimando created CALCITE-4301:
-
Summary: Unit test 'testCollectionsInnerValues()' for Cassandra
adapter is wrong
Key: CALCITE-4301
URL: https://issues.apache.org/jira/browse/CALCITE-4301
Project: Calcite
Issue Type: Test
Components: cassandra-adapter
Affects Versions: 1.25.0
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando
The following unit test included in
`cassandra/src/test/java/org/apache/calcite/test/CassandraAdapterDataTypesTest.java`
is wrong, as it applied _ITEM_ operator to a struct field _f_tuple_, instead
of the _DOT_ operator, returning _null_ instead of the correct _non-null_
values. __
{code:java}
@Test void testCollectionsInnerValues() {
CalciteAssert.that()
.with(DTCASSANDRA)
.query("select \"f_list\"[1], "
+ "\"f_map\"['k1'], "
+ "\"f_tuple\"['1'], "
+ "\"f_tuple\"['2'], "
+ "\"f_tuple\"['3']"
+ " from \"test_collections\"")
.returns("EXPR$0=1"
+ "; EXPR$1=v1"
+ "; EXPR$2=30"
+ "; EXPR$3=30ff87"
+ "; EXPR$4=2015-05-03 13:30:54.234");{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4293) cassandra adapter returns null when selecting non-null tuple elements
Alessandro Solimando created CALCITE-4293:
-
Summary: cassandra adapter returns null when selecting non-null
tuple elements
Key: CALCITE-4293
URL: https://issues.apache.org/jira/browse/CALCITE-4293
Project: Calcite
Issue Type: Bug
Components: cassandra-adapter
Affects Versions: 1.25.0
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando
The following test currently fails due to the _EXPR$i_ elements are null and
don't match their actual value within the _f_tuple_ field:
{code:java}
@Test void testTupleInnerValues() {
CalciteAssert.that()
.with(DTCASSANDRA)
.query("select x['1'], x['2'], x['3'] from "
+ "(select \"f_tuple\" from \"test_collections\") as T(x)")
.returns("EXPR$0=30"
+ "; EXPR$1=30ff87"
+ "; EXPR$2=2015-05-03 13:30:54\n");
}{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3465) Cover Cassandra 3.x data types
Alessandro Solimando created CALCITE-3465: - Summary: Cover Cassandra 3.x data types Key: CALCITE-3465 URL: https://issues.apache.org/jira/browse/CALCITE-3465 Project: Calcite Issue Type: Improvement Components: cassandra-adapter Affects Versions: 1.21.0 Reporter: Alessandro Solimando Currently cassandra-adapter covers only part of the [data types available in Cassandra 3.x|[https://docs.datastax.com/en/archived/cql/3.3/cql/cql_reference/cql_data_types_c.html]], the scope of the ticket is to extend the coverage and support all data types. Current status: ||CQL Data Type||SQL Data Type||Java Class||Supported|| |uuid|CHAR|java.lang.String|Y| |timeuuid|CHAR|java.lang.String|Y| |ascii|VARCHAR|java.lang.String|Y| |text|VARCHAR|java.lang.String|Y| |varchar|VARCHAR|java.lang.String|Y| |int (cint)|INTEGER|int|Y| |varint|INTEGER|int|Y| |bigint|BIGINT|long|Y| |double (cdouble)|DOUBLE|double|Y| |float (cfloat)|DOUBLE (should be REAL?)|float|Y| |decimal|DOUBLE|N/A|Y| |blob|VARBINARY|N/A|N| |boolean|BOOLEAN|N/A|N| |counter|INTEGER|N/A|N| |date|DATE|N/A|N| |frozen|?|N/A|N| |inet|?|N/A|N| |list|ARRAY?|N/A|N| |map|MAP|N/A|N| |set|?|N/A|N| |smallint|SMALLINT|N/A|N| |time|TIME?|N/A|N| |timestamp|TIMESTAMP|N/A|N| |tinyint|TINYINT|N/A|N| |tuple|ARRAY?|N/A|N| Second column is derived from _org.apache.calcite.adapter.cassandra.CassandraSchema.getRelDataType(...),_ third column from _org.apache.calcite.adapter.cassandra.currentRowField(...)._ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-2185) Additional unit tests for Spark Adapter
Alessandro Solimando created CALCITE-2185: - Summary: Additional unit tests for Spark Adapter Key: CALCITE-2185 URL: https://issues.apache.org/jira/browse/CALCITE-2185 Project: Calcite Issue Type: Test Components: spark Reporter: Alessandro Solimando Assignee: Julian Hyde Add some unit tests covering more aspects of the Spark Adapter. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2184) subquery leading to a "java.lang.ClassCastException: RexSubQuery cannot be cast to RexLocalRef"
Alessandro Solimando created CALCITE-2184:
-
Summary: subquery leading to a "java.lang.ClassCastException:
RexSubQuery cannot be cast to RexLocalRef"
Key: CALCITE-2184
URL: https://issues.apache.org/jira/browse/CALCITE-2184
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.16.0
Reporter: Alessandro Solimando
Assignee: Julian Hyde
Executing this query Calcite tries to cast the subquery (_RexSubQuery_) to a
local reference (_RexLocalRef_), resulting in a _ClassCastException_.
Here is the query:
{code:sql}
select *
from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
where exists (
select *
from (values (1, 'a'), (2, 'b')) as v(w, z)
where w < x
)
{code}
But the same happens with other (similar) queries, such as the following:
{code:sql}
select x
from (values (1, 'a'), (2, 'b')) as t(x, y)
where x <= all (
select x
from (values (1, 'a'), (2, 'b'), (1, 'b'), (2, 'c'), (2, 'c')) as t(x, y)
)
{code}
Please note that this problem is "masked" by the issues described in
[CALCITE-2183|https://issues.apache.org/jira/browse/CALCITE-2183], and it can
be reproduced only by "patching" this second.
Below the complete stack trace:
{noformat}
java.lang.RuntimeException: With materializationsEnabled=false, limit=0
at
org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1346)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1329)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returnsUnordered(CalciteAssert.java:1357)
at
org.apache.calcite.test.SparkAdapterTest.commonTester(SparkAdapterTest.java:93)
at
org.apache.calcite.test.SparkAdapterTest.testFilterExists(SparkAdapterTest.java:720)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.sql.SQLException: Error while executing SQL "select *
from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
where exists (
select *
from (values (1, 'a'), (2, 'b')) as v(w, z)
where w < x
)": org.apache.calcite.rex.RexSubQuery cannot be cast to
org.apache.calcite.rex.RexLocalRef
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
at
org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:568)
... 27 more
Caused by: java.lang.ClassCastException: org.apache.calcite.rex.RexSubQuery
cannot be cast to org.apache.calcite.rex.RexLocalRef
at
org.apache.calcite.rex.RexProgramBuilder.registerInput(RexProgramBuilder.java:298)
at
org.apache.calcite.rex.RexProgramBuilder.addCondition(RexProgramBuilder.java:272)
at
[jira] [Created] (CALCITE-2183) invocation of copy method in RelSubset class
Alessandro Solimando created CALCITE-2183:
-
Summary: invocation of copy method in RelSubset class
Key: CALCITE-2183
URL: https://issues.apache.org/jira/browse/CALCITE-2183
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.16.0
Reporter: Alessandro Solimando
Assignee: Julian Hyde
Consider the following query:
{code:sql}
select *
from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
where x between 3 and 4
{code}
When executed, an exception is thrown (the full stack trace is at the end) in
_copy_ method in _RelSubset_ class, as the method is not meant to be called.
This happens (in this particular case) while Calcite is trying to get rid of
unused terms (specifically, _trimUnusedFields_ method from _SqlToRelConverted_
class).
This is problematic as the trace of calls is legitimate, so the method should
be executable.
Complete stack trace documenting the issue:
{noformat}
java.lang.RuntimeException: With materializationsEnabled=false, limit=0
at
org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1346)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1329)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returnsUnordered(CalciteAssert.java:1357)
at
org.apache.calcite.test.SparkAdapterTest.commonTester(SparkAdapterTest.java:93)
at
org.apache.calcite.test.SparkAdapterTest.testFilterBetween(SparkAdapterTest.java:460)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.sql.SQLException: Error while executing SQL "select *
from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
where x between 3 and 4": null
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
at
org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:568)
... 27 more
Caused by: java.lang.UnsupportedOperationException
at org.apache.calcite.plan.volcano.RelSubset.copy(RelSubset.java:149)
at
org.apache.calcite.sql2rel.SqlToRelConverter.trimUnusedFields(SqlToRelConverter.java:517)
at org.apache.calcite.prepare.Prepare.trimUnusedFields(Prepare.java:391)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:304)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:230)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:781)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610)
at
[jira] [Created] (CALCITE-2145) Avatica tests fail depending on local timezone
Alessandro Solimando created CALCITE-2145:
-
Summary: Avatica tests fail depending on local timezone
Key: CALCITE-2145
URL: https://issues.apache.org/jira/browse/CALCITE-2145
Project: Calcite
Issue Type: Bug
Components: avatica
Affects Versions: avatica-1.10.0, avatica-1.11.0
Reporter: Alessandro Solimando
Running the tests runs correctly with timezone forced to UTC, but fail with
other timezones (verified with GMT+8 and GMT+2).
All methods using default locale (e.g., Locale.getDefault()) or timezone
(e.g., TimeZone.getDefault()) should have been forbidden (-CALCITE-1167-).
For instance, this command works:
{panel}
MAVEN_OPTS="$MAVEN_OPTS -Duser.timezone=UTC" mvn clean test
{panel}
This other command fails:
{panel}
MAVEN_OPTS="$MAVEN_OPTS -Duser.timezone=GMT+8" mvn clean test
{panel}
In what follows the relevant extract from console when running mvn test.
{panel:title=mvn clean test console output extract}
[...]
2018-01-22 01:56:02,633 [main] INFO - Creating table BATCH_EXECUTE_33
2018-01-22 01:56:02,635 [main] INFO - Creating table BATCH_EXECUTE_34
2018-01-22 01:56:02,675 [main] INFO - Creating table BATCH_CLEARS_36
2018-01-22 01:56:02,676 [main] INFO - Creating table BATCH_CLEARS_37
Tests run: 78, Failures: 2, Errors: 0, Skipped: 4, Time elapsed: 0.569 sec <<<
FAILURE! - in org.apache.calcite.avatica.RemoteDriverTest
testBatchInsertWithDates[JSON](org.apache.calcite.avatica.RemoteDriverTest)
Time elapsed: 0.013 sec <<< FAILURE!
java.lang.AssertionError: Wrong day for row 0 expected:<21> but was:<20>
at
org.apache.calcite.avatica.RemoteDriverTest.executeBatchInsertWithDates(RemoteDriverTest.java:1444)
at
org.apache.calcite.avatica.RemoteDriverTest.access$1100(RemoteDriverTest.java:91)
at
org.apache.calcite.avatica.RemoteDriverTest.eachConnection(RemoteDriverTest.java:228)
at
org.apache.calcite.avatica.RemoteDriverTest.testBatchInsertWithDates(RemoteDriverTest.java:1379)
testBatchInsertWithDates[PROTOBUF](org.apache.calcite.avatica.RemoteDriverTest)
Time elapsed: 0.003 sec <<< FAILURE!
java.lang.AssertionError: Wrong day for row 0 expected:<21> but was:<20>
at
org.apache.calcite.avatica.RemoteDriverTest.executeBatchInsertWithDates(RemoteDriverTest.java:1444)
at
org.apache.calcite.avatica.RemoteDriverTest.access$1100(RemoteDriverTest.java:91)
at
org.apache.calcite.avatica.RemoteDriverTest.eachConnection(RemoteDriverTest.java:228)
at
org.apache.calcite.avatica.RemoteDriverTest.testBatchInsertWithDates(RemoteDriverTest.java:1379)
[...]
Results :
Failed tests:
RemoteDriverTest.testBatchInsertWithDates:1379->eachConnection:228->access$1100:91->executeBatchInsertWithDates:1444
Wrong day for row 0 expected:<21> but was:<20>
RemoteDriverTest.testBatchInsertWithDates:1379->eachConnection:228->access$1100:91->executeBatchInsertWithDates:1444
Wrong day for row 0 expected:<21> but was:<20>
Tests run: 219, Failures: 2, Errors: 0, Skipped: 15
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Calcite Avatica Project . SUCCESS [ 2.233 s]
[INFO] Apache Calcite Avatica Metrics . SUCCESS [ 2.984 s]
[INFO] Apache Calcite Avatica . SUCCESS [ 26.595 s]
[INFO] Apache Calcite Avatica Server .. FAILURE [ 13.671 s]
[INFO] Apache Calcite Avatica Standalone Server ... SKIPPED
[INFO] Apache Calcite Avatica Docker images ... SKIPPED
[INFO] Apache Calcite Avatica Dropwizard Metrics 3 SKIPPED
[INFO] Apache Calcite Avatica Noop Driver . SKIPPED
[INFO] Apache Calcite Avatica Compatibility Kit ... SKIPPED
[INFO] Apache Calcite Avatica (Shaded) SKIPPED
[INFO]
[INFO] BUILD FAILURE
[INFO]
[...]
{panel}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
