[jira] [Created] (CALCITE-6018) Support nullable parameters in UDFs

2023-09-20 Thread Vladimir Sitnikov (Jira) 
Vladimir Sitnikov created CALCITE-6018:
--

 Summary: Support nullable parameters in UDFs
 Key: CALCITE-6018
 URL: https://issues.apache.org/jira/browse/CALCITE-6018
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Vladimir Sitnikov


Currently Calcite treats almost all the parameters as non-nullable. There's 
{{@Parameter(optional=true)}}, however, it mixes "optionality" vs "nullability".

It causes weird "cast(... NOT NULL)" even though my function processes null 
just fine.

See example in CALCITE-6012.

{noformat}
expr#4=[TO_HEAP_REFERENCE($t3)],
expr#5=[CAST($t4):JavaType(class java.lang.Object) NOT NULL],
expr#6=[retainedSize($t5)],
{noformat}

{{TO_HEAP_REFERENCE}} returns nullable {{HeapReference}} (Java class): 
https://github.com/vlsi/mat-calcite-plugin/blob/4d4aa2284eeec69bc51da0c2e769ded06ef9ab97/MatCalcitePlugin/src/com/github/vlsi/mat/calcite/schema/objects/HeapOperatorTable.java#L27-L28

{{retainedSize}} is declared as 
https://github.com/vlsi/mat-calcite-plugin/blob/4d4aa2284eeec69bc51da0c2e769ded06ef9ab97/MatCalcitePlugin/src/com/github/vlsi/mat/calcite/functions/HeapFunctions.java#L129

I tried adding {{@Parameter(optional=true)}} to {{retainedSize}}, and it does 
not remove the cast. It does not eliminate {{NOT NULL}} part of the cast.

---

In any case, I think it is not quite right to mix nullaness and optionally 
concepts, and Calcite should inter nullness from nullability annotations.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Possible bug found in SqlToRelConverter

2023-09-20 Thread Julian Hyde 
That sounds like a bug. Can you please log a jira case?

The fix might be in SqlValidatorImpl’s identifier expansion, even though the 
problem shows up in SqlToRelConverter.

> On Sep 20, 2023, at 12:39 PM, Jordan Hannel  wrote:
> 
> Hello all,
> I believe I have found a bug in SqlToRelConverter. The symptom and repro are 
> outlined below, and the full stacktrace is attached. This is on calcite 
> version 1.35.0. Any insight would be much appreciated, thanks!
> 
> When I parse the below SQL string to get a SqlNode, and call 
> SqlToRelConverter.convertQuery on this SqlNode, I get a 
> UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: 
> LEAD(COUNT(*)) OVER (ORDER BY `dim4`)
> 
> SELECT "dim4" AS "dim7",
> LEAD(COUNT(*), -1) OVER (ORDER BY "dim4") AS "measure10"
> FROM (
> SELECT "timestamp" AS "dim4"
> FROM "Shared.factDataset") AS "t0"
> GROUP BY "dim4"
> 
> I have confirmed that my SqlToRelConverter instance seems generally ok, 
> because it is able to convert many other SqlNodes to RelNodes.
> After some experimentation, I noticed that just changing `ORDER BY "dim4"` to 
> `ORDER BY "t0"."dim4"` fixes the issue - after that, the SqlNode successfully 
> is translated to a RelNode.
> 
> -- 
>  
> Jordie Hannel
> Founding Product Owner
> www.netspring.io

Possible bug found in SqlToRelConverter

2023-09-20 Thread Jordan Hannel 
Hello all,
I believe I have found a bug in SqlToRelConverter. The symptom and repro
are outlined below, and the full stacktrace is attached. This is on calcite
version 1.35.0. Any insight would be much appreciated, thanks!

When I parse the below SQL string to get a SqlNode, and call
SqlToRelConverter.convertQuery on this SqlNode, I get a
UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall:
LEAD(COUNT(*)) OVER (ORDER BY `dim4`)

SELECT "dim4" AS "dim7",
LEAD(COUNT(*), -1) OVER (ORDER BY "dim4") AS "measure10"
FROM (
SELECT "timestamp" AS "dim4"
FROM "Shared.factDataset") AS "t0"
GROUP BY "dim4"

I have confirmed that my SqlToRelConverter instance seems generally ok,
because it is able to convert many other SqlNodes to RelNodes.
After some experimentation, I noticed that just changing `ORDER BY "dim4"`
to `ORDER BY "t0"."dim4"` fixes the issue - after that, the SqlNode
successfully is translated to a RelNode.

-- 
[image: Download the Buyers Guide for Warehouse-Native Product Analytics]

*Jordie Hannel*
Founding Product Owner
www.netspring.io
java.lang.UnsupportedOperationException: class 
org.apache.calcite.sql.SqlBasicCall: LEAD(COUNT(*)) OVER (ORDER BY `dim4`)
at org.apache.calcite.util.Util.needToImplement(Util.java:1119)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.getValidatedNodeType(SqlValidatorImpl.java:1814)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertOver(SqlToRelConverter.java:2263)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.access$1900(SqlToRelConverter.java:235)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5461)
at 
org.apache.calcite.sql2rel.StandardConvertletTable.lambda$new$11(StandardConvertletTable.java:271)
at 
org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:59)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5655)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4826)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:166)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5468)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.createAggImpl(SqlToRelConverter.java:3499)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertAgg(SqlToRelConverter.java:3347)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:755)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:681)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3747)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:601)
at 
io.netspring.netscript.NetScriptSql.parseRelNode(NetScriptSql.java:224)
at 
io.netspring.netscript.NetScriptSql.convertDialect(NetScriptSql.java:354)
at io.netspring.polymon.net.SqlUtil.convertSqlDialect(SqlUtil.java:614)
at 
io.netspring.polymon.explore.ExploreQueryTest.testDialectSql(ExploreQueryTest.java:182)
at 
io.netspring.polymon.explore.ExploreQueryTest.testDialectSql(ExploreQueryTest.java:165)
at 
io.netspring.polymon.explore.ExploreQueryTest.runTest(ExploreQueryTest.java:154)
at 
io.netspring.polymon.explore.EventSegmentationQueryTest.runTest(EventSegmentationQueryTest.java:77)
at 
io.netspring.polymon.explore.EventSegmentationQueryTest.runTest(EventSegmentationQueryTest.java:73)
at 
io.netspring.polymon.explore.EventSegmentationQueryTest.testCustomPropWithPeriodOver(EventSegmentationQueryTest.java:506)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at 
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at

Re: PR CALCITE-5825 and CALCITE-5826 Review Request

2023-09-20 Thread LakeShen 
Thank you for opening this PR, I left a couple of comments.

Best,
LakeShen

Runkang He  于2023年9月20日周三 08:29写道：

> Hi community, I have submitted two PRs with CALCITE-5825 and CALCITE-5826
> to add some useful string functions.
>
> In CALCITE-5825, I have added the URL_ENCODE and URL_DECODE function, which
> is enabled in Spark library.
> In CALCITE-5826, I have added the FIND_IN_SET function, which is enabled in
> Hive and Spark library.
>
> If someone has time, please help me to review these two PRs, I would
> appreciate it very much.
>
> 1.CALCITE-5825:
> Jira: https://issues.apache.org/jira/browse/CALCITE-5825
> PR: https://github.com/apache/calcite/pull/3318
> 2.CALCITE-5826:
> Jira: https://issues.apache.org/jira/browse/CALCITE-5826
> PR: https://github.com/apache/calcite/pull/3317
>
> Best,
> Runkang He
>

[jira] [Created] (CALCITE-6017) The github link of released versions in doc may has no reference value

2023-09-20 Thread Ran Tao (Jira) 
Ran Tao created CALCITE-6017:


 Summary: The github link of released versions in doc may has no 
reference value
 Key: CALCITE-6017
 URL: https://issues.apache.org/jira/browse/CALCITE-6017
 Project: Calcite
  Issue Type: Improvement
Reporter: Ran Tao


In calcite history page, we have a description:

 
For a full list of releases, see
https://github.com/apache/calcite/releases;>github. 

however, this page has nothing, because calcite not use github to manage the 
released versions. So this description may seem a bit strange to end users.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)