Mark,
I have not seen your request for a Jira account. Can you please try
applying again here: https://issues.apache.org/jira/issues/Dashboard.jspa
Francis
On 16/10/2025 9:33 pm, Mark Lewis wrote:
Hi Ruben,
I don't have an Apache Jira account so am not able to raise a Jira ticket. I
did apply for an account yesterday using the self-serve form but still waiting.
I actually hit the failures running tests within substrait-java, so this is not
a completely standard configuration either:
https://github.com/substrait-io/substrait-java/blob/e0f453059e5c8d07e0cf303bd76d19158f21bed2/isthmus/src/test/java/io/substrait/isthmus/TpcdsQueryTest.java#L33
Here is the complete stacktrace for the failure on TPC-DS query 40:
Caused by: java.lang.RuntimeException: while converting CASE WHEN CAST(`D_DATE` AS
DATE) < CAST('2000-03-11' AS DATE) THEN `CS_SALES_PRICE` - CASE WHEN
`CR_REFUNDED_CASH` IS NOT NULL THEN `CR_REFUNDED_CASH` ELSE 0 END ELSE 0 END
at
org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerNodeTypeMethod$1(ReflectiveConvertletTable.java:99)
at
org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5930)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5093)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5744)
at
org.apache.calcite.sql2rel.AggConverter.translateAgg(AggConverter.java:471)
at
org.apache.calcite.sql2rel.AggConverter.translateAgg(AggConverter.java:338)
at org.apache.calcite.sql2rel.AggConverter.visit(AggConverter.java:322)
at org.apache.calcite.sql2rel.AggConverter$1.visit(AggConverter.java:177)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
at org.apache.calcite.sql2rel.AggConverter.visit(AggConverter.java:329)
at org.apache.calcite.sql2rel.AggConverter$1.visit(AggConverter.java:177)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
at
org.apache.calcite.sql2rel.SqlToRelConverter.createAggImpl(SqlToRelConverter.java:3587)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertAgg(SqlToRelConverter.java:3528)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:4767)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:811)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:737)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3943)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:630)
at
io.substrait.isthmus.calcite.rel.DdlSqlToRelConverter.handleNonDdl(DdlSqlToRelConverter.java:50)
at
io.substrait.isthmus.calcite.rel.DdlSqlToRelConverter.visit(DdlSqlToRelConverter.java:46)
at
io.substrait.isthmus.calcite.rel.DdlSqlToRelConverter.visit(DdlSqlToRelConverter.java:15)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
at
io.substrait.isthmus.sql.SubstraitSqlToCalcite.lambda$convert$0(SubstraitSqlToCalcite.java:140)
at
java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at
java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720)
at
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at
java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at
java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at
java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at
io.substrait.isthmus.sql.SubstraitSqlToCalcite.convert(SubstraitSqlToCalcite.java:147)
at
io.substrait.isthmus.sql.SubstraitSqlToCalcite.convertQueries(SubstraitSqlToCalcite.java:115)
at
io.substrait.isthmus.sql.SubstraitSqlToCalcite.convertQueries(SubstraitSqlToCalcite.java:89)
at io.substrait.isthmus.SqlToSubstrait.convert(SqlToSubstrait.java:56)
at
io.substrait.isthmus.PlanTestBase.toSubstraitPlan(PlanTestBase.java:356)
at
io.substrait.isthmus.TpcdsQueryTest.toSubstraitPlan(TpcdsQueryTest.java:61)
at
io.substrait.isthmus.TpcdsQueryTest.lambda$testQuery$0(TpcdsQueryTest.java:34)
at
org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:71)
... 52 more
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerNodeTypeMethod$1(ReflectiveConvertletTable.java:95)
... 92 more
Caused by: java.lang.RuntimeException: while converting CASE WHEN
`CR_REFUNDED_CASH` IS NOT NULL THEN `CR_REFUNDED_CASH` ELSE 0 END
at
org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerNodeTypeMethod$1(ReflectiveConvertletTable.java:99)
at
org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5930)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5093)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5744)
at
org.apache.calcite.sql2rel.StandardConvertletTable.convertOperands(StandardConvertletTable.java:1286)
at
org.apache.calcite.sql2rel.StandardConvertletTable.convertOperands(StandardConvertletTable.java:1278)
at
org.apache.calcite.sql2rel.StandardConvertletTable.convertCall(StandardConvertletTable.java:1184)
at
org.apache.calcite.sql2rel.StandardConvertletTable.lambda$new$2(StandardConvertletTable.java:163)
at
org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5930)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5093)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5744)
at
org.apache.calcite.sql2rel.StandardConvertletTable.convertCase(StandardConvertletTable.java:635)
... 97 more
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerNodeTypeMethod$1(ReflectiveConvertletTable.java:95)
... 112 more
Caused by: java.lang.UnsupportedOperationException: class
org.apache.calcite.sql.SqlBasicCall: `CR_REFUNDED_CASH`
at org.apache.calcite.util.Util.needToImplement(Util.java:1112)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.getValidatedNodeType(SqlValidatorImpl.java:1891)
at
org.apache.calcite.sql2rel.StandardConvertletTable.lambda$new$13(StandardConvertletTable.java:304)
at
org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5930)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5093)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5744)
at
org.apache.calcite.sql2rel.StandardConvertletTable.convertCase(StandardConvertletTable.java:635)
... 117 more
It might be that something is wrong in the code paths that the stacktrace shows
in the way the SqlNode objects are handled, but it seems that the problem stems
from the creation of the SqlNode structure, which is much more standard Calcite
usage:
https://github.com/substrait-io/substrait-java/blob/e0f453059e5c8d07e0cf303bd76d19158f21bed2/isthmus/src/main/java/io/substrait/isthmus/sql/SubstraitSqlStatementParser.java#L32-L35
Here we get the SqlBasicCall (with a cast operation) for the `CR_REFUNDED_CASH`
SqlIdentifier.
I will try to create a simpler reproduction to make diagnosis easier.
Regards,
Mark.
________________________________
From: Ruben Q L <[email protected]>
Sent: 16 October 2025 10:05
To: [email protected] <[email protected]>; [email protected]
<[email protected]>
Subject: Re: Regression in Calcite 1.41.0
@Mark, could you please provide more details for reproducibility? (details
about your system, Calcite configuration, full stack-trace, etc)
Also, if you're not already subscribed to the dev list, I encourage you to do
so (as explained in [1]) in order to be up-to-date with this (and other)
discussions.
Best,
Ruben
[1] https://calcite.apache.org/community/#mailing-lists
On Wed, Oct 15, 2025 at 5:03 PM Ruben Q L
<[email protected]<mailto:[email protected]>> wrote:
Definitely looks like a blocker.
As others have said, we would need a Jira ticket and, ideally, a minimalistic
unit test reproducing the issue.
On Wed, Oct 15, 2025 at 4:36 PM Mihai Budiu
<[email protected]<mailto:[email protected]>> wrote:
Sounds like a blocking problem.
Can you provide a reproduction? Calcite has many configuration settings.
In our tests we compile TPCDS without issues, but our configuration is not the
standard one.
Mihai
________________________________
From: 我 <[email protected]<mailto:[email protected]>>
Sent: Wednesday, October 15, 2025 8:34 AM
To: dev <[email protected]<mailto:[email protected]>>
Subject: Re: Regression in Calcite 1.41.0
Could you log this issue in Jira? It would be even better if you could also
provide a minimal test case!
Best,
Zhen Chen
---- Replied Message ----
| From | Mark Lewis<[email protected]<mailto:[email protected]>> |
| Date | 10/15/2025 22:36 |
| To | [email protected]<mailto:[email protected]> |
| Cc | |
| Subject | Regression in Calcite 1.41.0 |
Hi community,
I am making use of the Calcite SQLParser and SQLToRelSqlToRelConverter to
convert SQL statements from the TPC-DS benchmark suite. In preparation to adopt
the forthcoming Calcite 1.41.0, I tried using a version of Calcite I built
locally from the current development codebase. I see failures that did not
exist with Calcite 1.40.0 on TPC-DS queries 40, 67 and 80; all on CASE
statements, with the underlying failure in
SqlValidatorImpl.getValidatedNodeType():
Query 40 fails on:
CASE WHEN `CR_REFUNDED_CASH` IS NOT NULL THEN `CR_REFUNDED_CASH` ELSE 0 END
with the underlying error:
java.lang.UnsupportedOperationException: class
org.apache.calcite.sql.SqlBasicCall: `CR_REFUNDED_CASH`
Query 67 fails on:
CASE WHEN `SS_SALES_PRICE` * `SS_QUANTITY` IS NOT NULL THEN `SS_SALES_PRICE` *
`SS_QUANTITY` ELSE 0 END
with the underlying error:
java.lang.UnsupportedOperationException: class
org.apache.calcite.sql.SqlBasicCall: `SS_SALES_PRICE` * `SS_QUANTITY`
Query 80 fails on:
CASE WHEN `SR_RETURN_AMT` IS NOT NULL THEN `SR_RETURN_AMT` ELSE 0 END
with the underlying error:
ava.lang.UnsupportedOperationException: class
org.apache.calcite.sql.SqlBasicCall: `SR_RETURN_AMT`
This regression appears to be introduced in commit
12e7d621bbb19aa32cb45329cb84532115037b7a ([CALCITE-7044] Add internal operator
CAST NOT NULL to enhance rewrite COALESCE operator). Since that commit, the
SqlNode kind at the point of failure in each case has become SqlBasicCall with
a CAST NOT NULL operator, and no node type is identified, which is a failure
condition.
Regards,
Mark.
