[jira] [Created] (CALCITE-6112) Use inedible release tags
Vladimir Sitnikov created CALCITE-6112: -- Summary: Use inedible release tags Key: CALCITE-6112 URL: https://issues.apache.org/jira/browse/CALCITE-6112 Project: Calcite Issue Type: Improvement Reporter: Vladimir Sitnikov The ASF has recommended using inedible Git tags since 2016: https://lists.apache.org/thread/szbtzk0048ppx1zvzljbrq7by2mt1zvs It turns out that has broken in Calcite since 2020: https://github.com/apache/calcite/commit/2e30293af7373b6c5fbcc5fa6505b49df2fba000 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6019) Excessive cast when UDF receives nullable JavaType
Vladimir Sitnikov created CALCITE-6019:
--
Summary: Excessive cast when UDF receives nullable JavaType
Key: CALCITE-6019
URL: https://issues.apache.org/jira/browse/CALCITE-6019
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Vladimir Sitnikov
See CALCITE-6018, however, CALCITE-6018 is about the "ability to declare
nullable, non-optional parameter" while this issue is about the excessive NOT
NULL cast.
Just in case, adding {{@Strict}} to {{retainedSize}} does not make CAST go away.
It causes weird "cast(... NOT NULL)" even though my {{retainedSize}} function
processes null just fine.
{noformat}
expr#4=[TO_HEAP_REFERENCE($t3)],
expr#5=[CAST($t4):JavaType(class java.lang.Object) NOT NULL],
expr#6=[retainedSize($t5)],
{noformat}
{{TO_HEAP_REFERENCE}} returns nullable {{HeapReference}} (Java class):
[https://github.com/vlsi/mat-calcite-plugin/blob/4d4aa2284eeec69bc51da0c2e769ded06ef9ab97/MatCalcitePlugin/src/com/github/vlsi/mat/calcite/schema/objects/HeapOperatorTable.java#L27-L28]
{{retainedSize}} receives a single (nullable) {{Object}} parameter, and it is
declared as
[https://github.com/vlsi/mat-calcite-plugin/blob/4d4aa2284eeec69bc51da0c2e769ded06ef9ab97/MatCalcitePlugin/src/com/github/vlsi/mat/calcite/functions/HeapFunctions.java#L129]
I tried adding {{@Parameter(optional=true)}} to {{{}retainedSize{}}}, and it
does not eliminate {{NOT NULL}} part of the cast.
My expectation is that NOT NULL must not be there as both return type and the
parameter type are nullable. It might be cast should not be there as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6018) Support nullable parameters in UDFs
Vladimir Sitnikov created CALCITE-6018:
--
Summary: Support nullable parameters in UDFs
Key: CALCITE-6018
URL: https://issues.apache.org/jira/browse/CALCITE-6018
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Vladimir Sitnikov
Currently Calcite treats almost all the parameters as non-nullable. There's
{{@Parameter(optional=true)}}, however, it mixes "optionality" vs "nullability".
It causes weird "cast(... NOT NULL)" even though my function processes null
just fine.
See example in CALCITE-6012.
{noformat}
expr#4=[TO_HEAP_REFERENCE($t3)],
expr#5=[CAST($t4):JavaType(class java.lang.Object) NOT NULL],
expr#6=[retainedSize($t5)],
{noformat}
{{TO_HEAP_REFERENCE}} returns nullable {{HeapReference}} (Java class):
https://github.com/vlsi/mat-calcite-plugin/blob/4d4aa2284eeec69bc51da0c2e769ded06ef9ab97/MatCalcitePlugin/src/com/github/vlsi/mat/calcite/schema/objects/HeapOperatorTable.java#L27-L28
{{retainedSize}} is declared as
https://github.com/vlsi/mat-calcite-plugin/blob/4d4aa2284eeec69bc51da0c2e769ded06ef9ab97/MatCalcitePlugin/src/com/github/vlsi/mat/calcite/functions/HeapFunctions.java#L129
I tried adding {{@Parameter(optional=true)}} to {{retainedSize}}, and it does
not remove the cast. It does not eliminate {{NOT NULL}} part of the cast.
---
In any case, I think it is not quite right to mix nullaness and optionally
concepts, and Calcite should inter nullness from nullability annotations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6012) CAST($t4):JavaType(class java.lang.Object) NOT NULL causes NPE since 1.27
Vladimir Sitnikov created CALCITE-6012:
--
Summary: CAST($t4):JavaType(class java.lang.Object) NOT NULL
causes NPE since 1.27
Key: CALCITE-6012
URL: https://issues.apache.org/jira/browse/CALCITE-6012
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.27.0
Reporter: Vladimir Sitnikov
The commit that introduced the regression is
https://github.com/apache/calcite/commit/e819b4611e883c54708a75f6856300462c92b8ae
Here's UDF declaration:
https://github.com/vlsi/mat-calcite-plugin/blob/4d4aa2284eeec69bc51da0c2e769ded06ef9ab97/MatCalcitePlugin/src/com/github/vlsi/mat/calcite/schema/objects/HeapOperatorTable.java#L24-L36
Problematic stacktrace is below.
Frankly speaking, it is strange to see scaleIntervalToNumber in the stacktrace
as the types have nothing to do with intervals.
{noformat}
Caused by: java.lang.IllegalStateException: Unable to implement
EnumerableAggregate(group=[{}], retained_size=[SUM($0)]): rowcount = 8.0,
cumulative cost = {169.1038146973 rows, 721.0 cpu, 0.0 io}, id = 52
EnumerableCalc(expr#0=[{inputs}], expr#1=[0], expr#2=[GET_SNAPSHOT($t1)],
expr#3=[GET_IOBJECT($t2, $t0)], expr#4=[TO_HEAP_REFERENCE($t3)],
expr#5=[CAST($t4):JavaType(class java.lang.Object) NOT NULL],
expr#6=[retainedSize($t5)], $f0=[$t6]): rowcount = 80.0, cumulative cost =
{160.0 rows, 721.0 cpu, 0.0 io}, id = 54
EnumerableTableScan(table=[[HEAP, java, util, $ids$:HashMap]]): rowcount =
80.0, cumulative cost = {80.0 rows, 81.0 cpu, 0.0 io}, id = 36
at
org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:111)
at
org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:112)
at
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1124)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:311)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:208)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:642)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:508)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:478)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:213)
... 64 more
Suppressed: java.lang.NullPointerException
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.scaleIntervalToNumber(RexToLixTranslator.java:936)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.translateCast(RexToLixTranslator.java:592)
at
org.apache.calcite.adapter.enumerable.RexImpTable$CastImplementor.implementSafe(RexImpTable.java:2481)
at
org.apache.calcite.adapter.enumerable.RexImpTable$AbstractRexCallImplementor.genValueStatement(RexImpTable.java:2937)
at
org.apache.calcite.adapter.enumerable.RexImpTable$AbstractRexCallImplementor.implement(RexImpTable.java:2902)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:1134)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:90)
at org.apache.calcite.rex.RexCall.accept(RexCall.java:175)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:1018)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:90)
at
org.apache.calcite.rex.RexLocalRef.accept(RexLocalRef.java:75)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.implementCallOperand(RexToLixTranslator.java:1143)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:1130)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:90)
at org.apache.calcite.rex.RexCall.accept(RexCall.java:175)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:1018)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:90)
at
org.apache.calcite.rex.RexLocalRef.accept(RexLocalRef.java:75)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.translate(RexToLixTranslator.java:237)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.translate(RexToLixTranslator.java:231)
[jira] [Created] (CALCITE-6010) RelRule.Config requires too much ceremony for trivial cases
Vladimir Sitnikov created CALCITE-6010:
--
Summary: RelRule.Config requires too much ceremony for trivial
cases
Key: CALCITE-6010
URL: https://issues.apache.org/jira/browse/CALCITE-6010
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Vladimir Sitnikov
It takes a lot to implement RelRule.Config even for trivial cases when no extra
configuration options needed.
Sample:
https://github.com/apache/calcite/blob/d9dd3ac8a9f695e111a0a5e77f45b61b90f4b5b6/core/src/main/java/org/apache/calcite/rel/rules/TableScanRule.java#L72C35-L78
It requires users:
1) Use immutables or implement Config manually. Adding immutables processor
adds compile-time overhead, and implementing the interface manually is not
trivial
2) Implement CustomRule(Config) constructor
3) Implement Config.toRule() by calling CustomRule(Config)
---
I suggest:
1) Drop method Config.toRule(), and suggest users to call new ...Rule(config)
directly.
2) Provide default Config implementation along with Calcite. For instance
DefaultConfig.EMPTY, DefaultConfigBuilder...
3) Use composition for custom configurations, in other words, let custom rules
have their own attributes, and one of the attributes could be default config.
For instance: {{data class AggregateExpandDistinctAggregatesRule.Config(config
RelRule.Config, usingGroupingSets Boolean)}}
It would make it easier for users to implement config objects, and it would
reduce the code size (generated bytecode and native image size) as the current
Calcite approach duplicates the same "Config" methods like withOperandSupplier
across all the Config implementations
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5034) Disable Gradle build cache or use an alternative accout for it
Vladimir Sitnikov created CALCITE-5034: -- Summary: Disable Gradle build cache or use an alternative accout for it Key: CALCITE-5034 URL: https://issues.apache.org/jira/browse/CALCITE-5034 Project: Calcite Issue Type: Task Components: build Reporter: Vladimir Sitnikov Attachments: bucket-utilization-cache.csv Calcite uses build cache since CALCITE-4140. I'm paying for the service (~$6/month), and there's a risk bank will block further payments (I'm from Russia :-/ ), so I think it is time to switch to another account or disable the build-cache. I've chosen Wasabi S3 since they have a free egress policy, so the costs are predictable. Cloudflare R2 is not yet ready: https://blog.cloudflare.com/introducing-r2-object-storage/ Rough stats so far Total storage=15 GiB Upload=0.1-0.2 GiB/day Download=1-2 GiB/day -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (CALCITE-4889) Double join is created for NOT IN
Vladimir Sitnikov created CALCITE-4889:
--
Summary: Double join is created for NOT IN
Key: CALCITE-4889
URL: https://issues.apache.org/jira/browse/CALCITE-4889
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.28.0
Reporter: Vladimir Sitnikov
The following queries yield several joins in the plan.
I think double joins are excessive here, especially for the first case where
all the values are non-nullable.
{code}select * from "scott".emp where (empno, deptno) not in ((7369, 20),
(7499, 30));{code}
{noformat}
select * from "scott".emp where empno not in (null, 7782);
EnumerableCalc(expr#0..12=[{inputs}], expr#13=[0:BIGINT], expr#14=[=($t8,
$t13)], expr#15=[IS NULL($t12)], expr#16=[>=($t9, $t8)], expr#17=[AND($t15,
$t16)], expr#18=[OR($t14, $t17)], proj#0..7=[{exprs}], $condition=[$t18])
EnumerableMergeJoin(condition=[=($10, $11)], joinType=[left])
EnumerableSort(sort0=[$10], dir0=[ASC])
EnumerableCalc(expr#0..9=[{inputs}], proj#0..9=[{exprs}], EMPNO0=[$t0])
EnumerableNestedLoopJoin(condition=[true], joinType=[inner])
EnumerableTableScan(table=[[scott, EMP]])
EnumerableAggregate(group=[{}], agg#0=[COUNT()], agg#1=[COUNT($0)])
EnumerableValues(tuples=[[{ null }, { 7782 }]])
EnumerableSort(sort0=[$0], dir0=[ASC])
EnumerableCalc(expr#0=[{inputs}], expr#1=[true], proj#0..1=[{exprs}])
EnumerableValues(tuples=[[{ null }, { 7782 }]])
{noformat}
{noformat}
select * from "scott".emp where (empno, deptno) not in ((1, 2), (3, null));
EnumerableCalc(expr#0..14=[{inputs}], expr#15=[0:BIGINT], expr#16=[=($t8,
$t15)], expr#17=[IS NULL($t14)], expr#18=[>=($t9, $t8)], expr#19=[IS NOT
NULL($t11)], expr#20=[AND($t17, $t18, $t19)], expr#21=[OR($t16, $t20)],
proj#0..7=[{exprs}], $condition=[$t21])
EnumerableMergeJoin(condition=[AND(=($10, $12), =($11, $13))],
joinType=[left])
EnumerableSort(sort0=[$10], sort1=[$11], dir0=[ASC], dir1=[ASC])
EnumerableCalc(expr#0..9=[{inputs}], proj#0..9=[{exprs}], EMPNO0=[$t0],
DEPTNO0=[$t7])
EnumerableNestedLoopJoin(condition=[true], joinType=[inner])
EnumerableTableScan(table=[[scott, EMP]])
EnumerableAggregate(group=[{}], agg#0=[COUNT()], agg#1=[COUNT($0,
$1)])
EnumerableValues(tuples=[[{ 1, 2 }, { 3, null }]])
EnumerableSort(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC])
EnumerableCalc(expr#0..1=[{inputs}], expr#2=[true], proj#0..2=[{exprs}])
EnumerableValues(tuples=[[{ 1, 2 }, { 3, null }]])
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (CALCITE-4881) Calcite release tags should have rel/... prefix as per the ASF requirements
Vladimir Sitnikov created CALCITE-4881: -- Summary: Calcite release tags should have rel/... prefix as per the ASF requirements Key: CALCITE-4881 URL: https://issues.apache.org/jira/browse/CALCITE-4881 Project: Calcite Issue Type: Improvement Reporter: Vladimir Sitnikov As per "Git status update" mail from Infra, release commits should be tagged with rel/... tags which are protected at the Infra side so they can't be overwritten, removed, etc. See sample: ACCUMULO-4106 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (CALCITE-4832) Refactor Gradle to build-logic convention plugins
Vladimir Sitnikov created CALCITE-4832: -- Summary: Refactor Gradle to build-logic convention plugins Key: CALCITE-4832 URL: https://issues.apache.org/jira/browse/CALCITE-4832 Project: Calcite Issue Type: Improvement Components: build Reporter: Vladimir Sitnikov Assignee: Vladimir Sitnikov See https://docs.gradle.org/current/userguide/structuring_software_products.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4829) Bump Gradle to 7.2 and test with Java 17 at GitHub Actions
Vladimir Sitnikov created CALCITE-4829: -- Summary: Bump Gradle to 7.2 and test with Java 17 at GitHub Actions Key: CALCITE-4829 URL: https://issues.apache.org/jira/browse/CALCITE-4829 Project: Calcite Issue Type: Improvement Components: build, core Reporter: Vladimir Sitnikov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4824) ServerUnParserTest#testCreateTableVirtualColumn fails
Vladimir Sitnikov created CALCITE-4824:
--
Summary: ServerUnParserTest#testCreateTableVirtualColumn fails
Key: CALCITE-4824
URL: https://issues.apache.org/jira/browse/CALCITE-4824
Project: Calcite
Issue Type: Improvement
Components: server
Reporter: Vladimir Sitnikov
It looks like {{org.apache.calcite.server.ServerDdlExecutor#INSTANCE}} results
in non-synchronized HashMap instances in ReflectUtil.
Sample failure:
{noformat}
ServerUnParserTest > testCreateTableVirtualColumn() STANDARD_ERROR
log4j:WARN No appenders could be found for logger
(org.apache.calcite.sql.parser).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
WARNING 2.8sec, 442 completed, 0 failed, 5 skipped,
org.apache.calcite.test.ServerParserTest
WARNING 3.8sec, 442 completed, 0 failed, 6 skipped,
org.apache.calcite.test.ServerUnParserTest
2.2sec, org.apache.calcite.test.ServerTest > testVirtualColumn()
org.apache.calcite.test.ServerQuidemTest > test(String)[3], [3] sql\table.iq
failure marker
FAILURE 3.6sec, org.apache.calcite.test.ServerQuidemTest > test(String)[3],
[3] sql\table.iq
org.opentest4j.AssertionFailedError: Files differ:
D:\a\calcite\calcite\server\build\resources\test\sql\surefire\sql\table.iq
D:\a\calcite\calcite\server\build\resources\test\sql\table.iq
23c23,155
< (0 rows modified)
---
> java.sql.SQLException: Error while executing SQL "create table t (i int,
j int not null)": Method not found: execute([class
org.apache.calcite.sql.SqlNode, interface
org.apache.calcite.jdbc.CalcitePrepare$Context])
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
> at
org.apache.calcite.avatica.AvaticaStatement.executeLargeUpdate(AvaticaStatement.java:246)
> at
org.apache.calcite.avatica.AvaticaStatement.executeUpdate(AvaticaStatement.java:240)
> at net.hydromatic.quidem.Quidem.update(Quidem.java:279)
> at net.hydromatic.quidem.Quidem.access$2900(Quidem.java:54)
> at net.hydromatic.quidem.Quidem$ContextImpl.update(Quidem.java:1752)
> at net.hydromatic.quidem.Quidem$UpdateCommand.execute(Quidem.java:1152)
> at
net.hydromatic.quidem.Quidem$CompositeCommand.execute(Quidem.java:1548)
> at net.hydromatic.quidem.Quidem.execute(Quidem.java:216)
> at org.apache.calcite.test.QuidemTest.checkRun(QuidemTest.java:156)
> at
org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
> at
org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.compute(ForkJoinPoolHierarchicalTestExecutorService.java:185)
> at java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
> at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
> at
java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
> at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
> at
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
> Caused by: java.lang.IllegalArgumentException: Method not found:
execute([class org.apache.calcite.sql.SqlNode, interface
org.apache.calcite.jdbc.CalcitePrepare$Context])
> at
org.apache.calcite.util.ReflectUtil$2.lookupMethod(ReflectUtil.java:563)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
org.apache.calcite.server.DdlExecutorImpl.executeDdl(DdlExecutorImpl.java:41)
> at
org.apache.calcite.prepare.CalcitePrepareImpl.executeDdl(CalcitePrepareImpl.java:369)
> at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:634)
> at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:513)
> at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:483)
> at
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:249)
> at
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)
> at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
> at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> ... 116 more
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4823) JEP 411: AccessControlException, AccessController in java.security has been deprecated and marked for removal
Vladimir Sitnikov created CALCITE-4823:
--
Summary: JEP 411: AccessControlException, AccessController in
java.security has been deprecated and marked for removal
Key: CALCITE-4823
URL: https://issues.apache.org/jira/browse/CALCITE-4823
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Vladimir Sitnikov
Calcite uses java.security APIs that have been marked for removal.
Let's minimize the usages of java.security for now.
{noformat}
/Users/runner/work/calcite/calcite/core/src/main/java/org/apache/calcite/util/SaffronProperties.java:28:
warning: [removal] AccessControlException in java.security has been deprecated
and marked for removal
import java.security.AccessControlException;
^
/Users/runner/work/calcite/calcite/core/src/main/java/org/apache/calcite/config/CalciteSystemProperty.java:27:
warning: [removal] AccessControlException in java.security has been deprecated
and marked for removal
import java.security.AccessControlException;
^
error: warnings found and -Werror specified
/Users/runner/work/calcite/calcite/core/src/main/java/org/apache/calcite/runtime/Resources.java:991:
warning: [removal] AccessController in java.security has been deprecated and
marked for removal
return java.security.AccessController.doPrivileged(
^
/Users/runner/work/calcite/calcite/core/src/main/java/org/apache/calcite/util/SaffronProperties.java:137:
warning: [removal] AccessControlException in java.security has been deprecated
and marked for removal
} catch (AccessControlException e) {
^
/Users/runner/work/calcite/calcite/core/src/main/java/org/apache/calcite/config/CalciteSystemProperty.java:439:
warning: [removal] AccessControlException in java.security has been deprecated
and marked for removal
} catch (AccessControlException e) {
^
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
1 error
5 warnings
:core:compileJava failure marker
{noformat}
What do we do with that?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4821) Move utility test classes into calcite-testkit and unpublish -test.jar
Vladimir Sitnikov created CALCITE-4821:
--
Summary: Move utility test classes into calcite-testkit and
unpublish -test.jar
Key: CALCITE-4821
URL: https://issues.apache.org/jira/browse/CALCITE-4821
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Vladimir Sitnikov
-test.jar make it hard to support build configuration, and it blocks migration
to Gradle 7.
As we move the commonly used classes to its own -testkit module, it would make
it easier to reuse test helpers, it would make easier to recognize the Test API
vs {{@Test}} code, and it would make certain classes smaller. E.g.
CatchAllSchema is currently located in ReflectiveSchemaTest, and if we move it
to a top-level class, then ReflectiveSchemaTest would be smaller.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4715) Create CITATION.cff for BibTex, APA citation UI in GitHub
Vladimir Sitnikov created CALCITE-4715: -- Summary: Create CITATION.cff for BibTex, APA citation UI in GitHub Key: CALCITE-4715 URL: https://issues.apache.org/jira/browse/CALCITE-4715 Project: Calcite Issue Type: Improvement Reporter: Vladimir Sitnikov See https://twitter.com/natfriedman/status/1420122675813441540 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4701) Implement and publish Gradle plugin for extending the parser
Vladimir Sitnikov created CALCITE-4701: -- Summary: Implement and publish Gradle plugin for extending the parser Key: CALCITE-4701 URL: https://issues.apache.org/jira/browse/CALCITE-4701 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.27.0 Reporter: Vladimir Sitnikov Currently, calcite-core.jar contains the relevant parser template files, however, the usage of the templates is not clear from the client perspective (especially when you are new to Calcite). It would be nice if Calcite published a plugin that simplifies building a parser extension. We could reuse the same plugin in calcite-babel module. Currently babel references $rootDir/core/... directrly rather than fetching the parser templates from core jar. It might make sense to ship parser templates in its own jar file, however, I have no strong opinion here. Parser.jj consumes <1% of the compressed jar size(225K compresses to 44K), so it is not a big deal, yet having a meaningful artifact name for the parser templates looks like the right thing to do. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4679) Search/sarg simplification leaves is not null(literal) unsimplified
Vladimir Sitnikov created CALCITE-4679:
--
Summary: Search/sarg simplification leaves is not null(literal)
unsimplified
Key: CALCITE-4679
URL: https://issues.apache.org/jira/browse/CALCITE-4679
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Vladimir Sitnikov
Case: {{SEARCH(100500, Sarg[=])}} (Sarg[=] is "all values match, unknown as
unknown").
While the issue does not look severe, it results in rex fuzzer false positives,
so it might mask true bugs.
Expected: {{true}}
Actual:
{noformat}
unknown as unknown: OR(IS NOT NULL(0), null)
unknown as false: IS NOT NULL(0)
unknown as true: true <-- finally got the right answer
{noformat}
Test:
{code:java}
checkSimplify(
rexBuilder.makeCall(
SqlStdOperatorTable.SEARCH,
literal(BigDecimal.ZERO),
rexBuilder.makeSearchArgumentLiteral(
Sarg.of(RexUnknownAs.UNKNOWN,
ImmutableRangeSet.of(Range.all())),
tInt())),
"true"
);
{code}
---
fuzzer:
{code:java}
@Test void singleFuzzyTest() {
Random r = new Random();
r.setSeed(-8889103384303613092L);
RexFuzzer fuzzer = new RexFuzzer(rexBuilder, typeFactory);
generateRexAndCheckTrueFalse(fuzzer, r);
}{code}
{noformat}
AssertionFailedError: SEARCH(100500, Sarg[=])
rexBuilder.makeCall(SqlStdOperatorTable.SEARCH, literal(100500),
literal(Sarg[=])) isAlwaysTrue, so it should simplify to TRUE unknownAsFalse
==> expected: but was:
at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
at
org.junit.jupiter.api.AssertionUtils.failNotEqual(AssertionUtils.java:69)
at
org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:188)
at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1146)
at
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.checkUnknownAs(RexProgramFuzzyTest.java:251)
at
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.checkUnknownAsAndShrink(RexProgramFuzzyTest.java:202)
at
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.checkUnknownAs(RexProgramFuzzyTest.java:165)
at
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.generateRexAndCheckTrueFalse(RexProgramFuzzyTest.java:454)
at
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.singleFuzzyTest(RexProgramFuzzyTest.java:463)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4678) AssertionError: result mismatch when simplifying case+search+isdistinctfrom+isnottrue
Vladimir Sitnikov created CALCITE-4678:
--
Summary: AssertionError: result mismatch when simplifying
case+search+isdistinctfrom+isnottrue
Key: CALCITE-4678
URL: https://issues.apache.org/jira/browse/CALCITE-4678
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Vladimir Sitnikov
Frankly speaking, I do not know what triggers the issue. so the issue summary
is abstract for now.
I do not know the expected outcome, however, I expect that Calcite should not
fail with AssertionErrors.
Test case:
{code:java}
checkSimplifyAs(
isNotTrue(
case_(
rexBuilder.makeCall(
SqlStdOperatorTable.SEARCH,
vInt(1),
rexBuilder.makeSearchArgumentLiteral(
Sarg.of(
RexUnknownAs.TRUE,
ImmutableRangeSet.of(Range.lessThan(BigDecimal.ZERO))),
tInt())),
trueLiteral,
le(trueLiteral, isDistinctFrom(literal(0), literal(0),
RexUnknownAs.TRUE,
is("no idea what is expected"));
{code}
Error:
{noformat}
result mismatch (unknown as TRUE): when applied to {?0.int1=NULL},
IS NOT TRUE(CASE(SEARCH(?0.int1, Sarg[(-∞..0); NULL AS TRUE]), true, <=(true,
IS DISTINCT FROM(0, 0 yielded false;
AND(IS NOT DISTINCT FROM(0, 0), >=(?0.int1, 0)) yielded true
java.lang.AssertionError: result mismatch (unknown as TRUE): when applied to
{?0.int1=NULL},
IS NOT TRUE(CASE(SEARCH(?0.int1, Sarg[(-∞..0); NULL AS TRUE]), true, <=(true,
IS DISTINCT FROM(0, 0 yielded false;
AND(IS NOT DISTINCT FROM(0, 0), >=(?0.int1, 0)) yielded true
at org.apache.calcite.rex.RexSimplify.verify(RexSimplify.java:2098)
at
org.apache.calcite.rex.RexSimplify.simplifyUnknownAs(RexSimplify.java:250)
at
org.apache.calcite.rex.RexProgramTestBase.checkSimplifyAs(RexProgramTestBase.java:152){noformat}
The relevant fuzzer-driven case is
{code:java}
@Test void singleFuzzyTest() {
Random r = new Random();
r.setSeed(6192825841324574146L);
RexFuzzer fuzzer = new RexFuzzer(rexBuilder, typeFactory);
generateRexAndCheckTrueFalse(fuzzer, r);
}{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4659) Introduce AssertJ and gradually migrate to its usage instead of Hamcrest
Vladimir Sitnikov created CALCITE-4659:
--
Summary: Introduce AssertJ and gradually migrate to its usage
instead of Hamcrest
Key: CALCITE-4659
URL: https://issues.apache.org/jira/browse/CALCITE-4659
Project: Calcite
Issue Type: Improvement
Reporter: Vladimir Sitnikov
AssertJ has already been discussed in CALCITE-2457, yet it was forgotten.
Note: I do not suggest replacing all the assertions with AssertJ, especially
"compare with golden output" ones.
The current issues with {{assertThat}}-based tests:
1) {{asertTrue}}, {{assertEquals(Object, Object)}} are prone to mistakes so one
can get "expected vs actual wrong".
2) Hamcrest matches are hard to discover. In other words, one cant use
autocomplete menu in IDE to see which other matches are available.
3) Hamcrest matches are hard to extend/implement. For instance, we have very
limited amount of Hamcrest-based matches, and we do have a lot of
Calcite-specific assertions.
AssertJ solves the issue of API discoverability (one writes
{{assertThat(...).containsExactly("skippedTest")}}), and AssertJ has a
well-known path for extensibility. We can implement Calcite-specific assertions
with AssertJ style, so the tests would be more consistent.
Refaster templates (see CALCITE-4658) might help to perform automatic
conversion from JUnit-style assertions to AssertJ.
There are templates to cleanup AssertJ assertions:
https://github.com/palantir/assertj-automation
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4658) Introduce refaster for automatic reformat of the code
Vladimir Sitnikov created CALCITE-4658:
--
Summary: Introduce refaster for automatic reformat of the code
Key: CALCITE-4658
URL: https://issues.apache.org/jira/browse/CALCITE-4658
Project: Calcite
Issue Type: Improvement
Reporter: Vladimir Sitnikov
Sometimes we have API usage rules which require manual actions to correct.
For instance, we "forbid" the use of {{Lists#newArrayList()}}, and suggest
replacing it with {{new ArrayList()}}.
It would be better if the build script could perform such replacements
automatically.
Refaster is a smart refactoring library for Java:
https://errorprone.info/docs/refaster, and it could help us to keep the code up
to date.
Of course, forbidden-apis checks might still be there (e.g. for those who
prefer manual replacements), however, adding automatic code cleanups would be
great.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4657) Replace cancel-workflow-runs with GitHub in-core cancel-in-progress feature
Vladimir Sitnikov created CALCITE-4657: -- Summary: Replace cancel-workflow-runs with GitHub in-core cancel-in-progress feature Key: CALCITE-4657 URL: https://issues.apache.org/jira/browse/CALCITE-4657 Project: Calcite Issue Type: Improvement Reporter: Vladimir Sitnikov See https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#concurrency https://github.com/potiuk/cancel-workflow-runs is deprecated -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4618) Refine dependency declarations to better account for type annotations
Vladimir Sitnikov created CALCITE-4618: -- Summary: Refine dependency declarations to better account for type annotations Key: CALCITE-4618 URL: https://issues.apache.org/jira/browse/CALCITE-4618 Project: Calcite Issue Type: Improvement Affects Versions: 1.26.0 Reporter: Vladimir Sitnikov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4612) Fuzzer for RelBuilder
Vladimir Sitnikov created CALCITE-4612: -- Summary: Fuzzer for RelBuilder Key: CALCITE-4612 URL: https://issues.apache.org/jira/browse/CALCITE-4612 Project: Calcite Issue Type: Test Components: core Reporter: Vladimir Sitnikov A generator of random rels would be useful for testing: a) Generate rel, write it down as SQL (ensures "Rel to SQL" does not fail), parse it back (ensures parser works OK), plans the tree (ensures the rules do not throw exceptions) b) Generate rel, serialize with one of the adapters (e.g. serialize to ElasticSearch). c) Generate rel, and execute it with https://github.com/sqlancer/sqlancer (or alike) approach to verify the results of the query. It might surface **correctness** bugs in optimizer, not just NPEs -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4598) Forbid the use of System.out
Vladimir Sitnikov created CALCITE-4598: -- Summary: Forbid the use of System.out Key: CALCITE-4598 URL: https://issues.apache.org/jira/browse/CALCITE-4598 Project: Calcite Issue Type: Improvement Affects Versions: 1.26.0 Reporter: Vladimir Sitnikov System.out.println often results in useless logging which is impossible to disable in the client code. We should remove all the uses of System.out.println and add something like jdk-system-out from forbiddenApis or https://errorprone.info/bugpattern/SystemOut I'm inclined to errorprone since it provides a clear way to suppress the warning (add annotation). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4595) Split DiffRepository test data into per-test files
Vladimir Sitnikov created CALCITE-4595: -- Summary: Split DiffRepository test data into per-test files Key: CALCITE-4595 URL: https://issues.apache.org/jira/browse/CALCITE-4595 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.26.0 Reporter: Vladimir Sitnikov The current issues: * the file is 13'000 lines, so it is hard to browse in IDE * contributions might result in merge conflicts, and "sorting the items" requires non-standard procedure (e.g. Calcite-specific sorting of the calcite-specific XML files) If we split the file into several smaller ones, then: * The number of merge conflicts would be greatly smaller * We won't need to sort the items inside the files. * At best, it would be compatible with https://approvaltests.com/ which has IDE plugins to automatically update the reference files * plan changes in GitHub PR would convey the test name right in the filename, so the review would be easier The drawbacks of having more files might be: a) Each file might have to have a license header that consumes ~16 lines. We could either blindly accept it or exclude it from RAT check. b) Global plan changes would touch lots of XML files. Currently, we copy a couple of files from "actual to expected", and with the new approach, we might need to copy more files. However, copying several files is not that hard. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4558) Sort CPU cost should be aligned with filter and project: sort should incur row copy cost
Vladimir Sitnikov created CALCITE-4558: -- Summary: Sort CPU cost should be aligned with filter and project: sort should incur row copy cost Key: CALCITE-4558 URL: https://issues.apache.org/jira/browse/CALCITE-4558 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.26.0 Reporter: Vladimir Sitnikov Typical Java implementations of the sort do not copy rows (they copy references only), so it makes little sense to have "row width" as the key driver of the sort costing. The CPU cost for filter does not include "row copy" cost. Even though the implementations might be different, in-core costs should be aligned. For instance, the current, EnumerableLimitSort and EnumerableSort have bytesPerRow multiplier, however, the implementation does not copy rows field-by-field . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4557) Sort costing should account the number of sort keys
Vladimir Sitnikov created CALCITE-4557: -- Summary: Sort costing should account the number of sort keys Key: CALCITE-4557 URL: https://issues.apache.org/jira/browse/CALCITE-4557 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.26.0 Reporter: Vladimir Sitnikov Current costing for sort does not account for the number of keys in the comparison function which might result in worse plan selection. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4553) Use Gradle Toolchains for customizing JVM used for build and test Calcite
Vladimir Sitnikov created CALCITE-4553: -- Summary: Use Gradle Toolchains for customizing JVM used for build and test Calcite Key: CALCITE-4553 URL: https://issues.apache.org/jira/browse/CALCITE-4553 Project: Calcite Issue Type: Improvement Affects Versions: 1.26.0 Reporter: Vladimir Sitnikov Gradle Toolchains can use different JVM for Gradle execution and for the build process. It might help for testing newer Javas which are not yet supported for Gradle execution. For instance, it can be "use the current JVM by default", and "use -PcalciteJavaHome=... JVM" if the property is added. Toolchains can be auto-provisioned, so it might simplify testing for the end-users. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4531) Deprecate RexLiteral#intValue since it performs silent truncation
Vladimir Sitnikov created CALCITE-4531:
--
Summary: Deprecate RexLiteral#intValue since it performs silent
truncation
Key: CALCITE-4531
URL: https://issues.apache.org/jira/browse/CALCITE-4531
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Vladimir Sitnikov
{{RexLiteral#intValue}} is prone to errors since it silently truncates values
to {{int}} so the developers might fail to know that at the compile time.
It might be safer to expose {{BigDecimal}} or {{intValueExact}} or
{{doubleValue}} alternatives which would be "enough for all the possible cases".
An alternative option is to mark the method as deprecated, so every use of the
method would require users to suppress the warning, so they know why the method
is deprecated.
I guess the most common use case for {{RexLiteral.intValue}} is {{offset}} and
{{fetch}} in {{Sort}}, however, the misuse is hard to spot, and it might result
in hard to notice data corruptions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4517) $node isAlwaysTrue, so it should simplify to TRUE unknownAsFalse
Vladimir Sitnikov created CALCITE-4517:
--
Summary: $node isAlwaysTrue, so it should simplify to TRUE
unknownAsFalse
Key: CALCITE-4517
URL: https://issues.apache.org/jira/browse/CALCITE-4517
Project: Calcite
Issue Type: Sub-task
Components: core
Reporter: Vladimir Sitnikov
Fix For: 1.27.0
{noformat}
@Test void singleFuzzyTest() {
Random r = new Random();
r.setSeed(6321443803263498676L);
RexFuzzer fuzzer = new RexFuzzer(rexBuilder, typeFactory);
generateRexAndCheckTrueFalse(fuzzer, r);
}
yields
$node isAlwaysTrue, so it should simplify to TRUE unknownAsFalse
SEARCH(-(COALESCE(?0.int0, CASE(=(CASE(false, SEARCH(?0.notNullInt0,
Sarg[(0..2]; NULL AS FALSE]), true), NOT(IS NOT TRUE(false))), -(+(100500),
-(CASE(true, 1, 100500), CASE(=(?0.notNullBool0, ?0.notNullBool0),
null:INTEGER, ?0.notNullInt1))), CASE(=(COALESCE(0, ?0.int1), 1), 1,
=(COALESCE(0, ?0.int1), -(null, ?0.int1)), CASE(?0.notNullBool0,
COALESCE(?0.int0, ?0.int1), -(null:INTEGER, null:INTEGER)), null:INTEGER)),
+(*(COALESCE(COALESCE(1, -2038957448, 0), CASE(false, ?0.int0, null:INTEGER),
COALESCE(?0.notNullInt0, null:INTEGER, null:INTEGER, -1), COALESCE(0,
933814882, ?0.int0)), COALESCE(+(100500), CASE(?0.bool1, ?0.notNullInt0,
null:INTEGER), CASE(=(-(null:INTEGER), 100500),
COALESCE(-(-(CASE(=(?0.bool0, ?0.notNullBool1), ?0.notNullInt1, null:INTEGER),
?0.notNullInt0)), +(COALESCE(-(-1), ?0.notNullInt1, -(null:INTEGER,
null:INTEGER))), CASE(=(OR(?0.bool0, ?0.bool1, false), <(true,
?0.notNullBool0)), -(+(null:INTEGER)), =(OR(?0.bool0, ?0.bool1, false),
SEARCH(null:INTEGER, Sarg[TRUE])), CASE(=(-1, null), CASE(=(?0.notNullBool1,
?0.bool0), 1135316986, null:INTEGER), =(-1, 1), +(null:INTEGER), =(-1, 100500),
-(1), +(?0.notNullInt1, ?0.notNullInt0)), COALESCE(-(null:INTEGER), CASE(true,
100500, null:INTEGER), *(1994412942, null))), CASE(=(IS
UNKNOWN(?0.notNullBool0), IS NOT UNKNOWN(?0.notNullBool1)), +(?0.notNullInt1),
=(IS UNKNOWN(?0.notNullBool0), IS DISTINCT FROM(?0.notNullInt1, 1)), -(null,
+(-1)), =(IS UNKNOWN(?0.notNullBool0), >(false, true)), +(-(null, ?0.int1),
*(100500, ?0.int0)), CASE(=(?0.bool0, false), +(-1, 100500), *(-1236110720,
?0.int0, =(-(null:INTEGER), +(+(COALESCE(?0.int0, null:INTEGER,
null:INTEGER, ?0.notNullInt0, ?0.notNullInt1, 100500)), Sarg[=])
Actual :IS NOT NULL(-(COALESCE(?0.int0, CASE(=(CASE(false,
SEARCH(?0.notNullInt0, Sarg[(0..2]; NULL AS FALSE]), true), NOT(IS NOT
TRUE(false))), -(+(100500), -(CASE(true, 1, 100500), CASE(=(?0.notNullBool0,
?0.notNullBool0), null:INTEGER, ?0.notNullInt1))), CASE(=(COALESCE(0, ?0.int1),
1), 1, =(COALESCE(0, ?0.int1), -(null, ?0.int1)), CASE(?0.notNullBool0,
COALESCE(?0.int0, ?0.int1), -(null:INTEGER, null:INTEGER)), null:INTEGER)),
+(*(COALESCE(COALESCE(1, -2038957448, 0), CASE(false, ?0.int0, null:INTEGER),
COALESCE(?0.notNullInt0, null:INTEGER, null:INTEGER, -1), COALESCE(0,
933814882, ?0.int0)), COALESCE(+(100500), CASE(?0.bool1, ?0.notNullInt0,
null:INTEGER), CASE(=(-(null:INTEGER), 100500),
COALESCE(-(-(CASE(=(?0.bool0, ?0.notNullBool1), ?0.notNullInt1, null:INTEGER),
?0.notNullInt0)), +(COALESCE(-(-1), ?0.notNullInt1, -(null:INTEGER,
null:INTEGER))), CASE(=(OR(?0.bool0, ?0.bool1, false), <(true,
?0.notNullBool0)), -(+(null:INTEGER)), =(OR(?0.bool0, ?0.bool1, false),
SEARCH(null:INTEGER, Sarg[TRUE])), CASE(=(-1, null), CASE(=(?0.notNullBool1,
?0.bool0), 1135316986, null:INTEGER), =(-1, 1), +(null:INTEGER), =(-1, 100500),
-(1), +(?0.notNullInt1, ?0.notNullInt0)), COALESCE(-(null:INTEGER), CASE(true,
100500, null:INTEGER), *(1994412942, null))), CASE(=(IS
UNKNOWN(?0.notNullBool0), IS NOT UNKNOWN(?0.notNullBool1)), +(?0.notNullInt1),
=(IS UNKNOWN(?0.notNullBool0), IS DISTINCT FROM(?0.notNullInt1, 1)), -(null,
+(-1)), =(IS UNKNOWN(?0.notNullBool0), >(false, true)), +(-(null, ?0.int1),
*(100500, ?0.int0)), CASE(=(?0.bool0, false), +(-1, 100500), *(-1236110720,
?0.int0, =(-(null:INTEGER), +(+(COALESCE(?0.int0, null:INTEGER,
null:INTEGER, ?0.notNullInt0, ?0.notNullInt1, 100500)))
at
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.checkUnknownAs(RexProgramFuzzyTest.java:251){noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4516) ...had non-nullable type BOOLEAN, and it was optimized to..that has nullable type BOOLEAN
Vladimir Sitnikov created CALCITE-4516:
--
Summary: ...had non-nullable type BOOLEAN, and it was optimized
to..that has nullable type BOOLEAN
Key: CALCITE-4516
URL: https://issues.apache.org/jira/browse/CALCITE-4516
Project: Calcite
Issue Type: Sub-task
Components: core
Reporter: Vladimir Sitnikov
Fix For: 1.27.0
{noformat}
COALESCE(SEARCH(CASE(true, -1076415185, null:INTEGER), Sarg[(0..+∞); NULL AS
TRUE]), AND(IS TRUE(?0.notNullBool0), =(?0.bool0, ?0.bool0), SEARCH(0,
Sarg[4])))
coalesce(rexBuilder.makeCall(SqlStdOperatorTable.SEARCH, case_(trueLiteral,
literal(-1076415185), nullInt), literal(Sarg[(0..+∞); NULL AS TRUE])),
and(isTrue(vBoolNotNull(0)), eq(vBool(0), vBool(0)),
rexBuilder.makeCall(SqlStdOperatorTable.SEARCH, literal(0), literal(Sarg[4]
had non-nullable type BOOLEAN, and it was optimized to >(CASE(true,
-1076415185, null:INTEGER), 0)
gt(case_(trueLiteral, literal(-1076415185), nullInt), literal(0)) that has
nullable type BOOLEAN
org.opentest4j.AssertionFailedError: COALESCE(SEARCH(CASE(true, -1076415185,
null:INTEGER), Sarg[(0..+∞); NULL AS TRUE]), AND(IS TRUE(?0.notNullBool0),
=(?0.bool0, ?0.bool0), SEARCH(0, Sarg[4])))
coalesce(rexBuilder.makeCall(SqlStdOperatorTable.SEARCH, case_(trueLiteral,
literal(-1076415185), nullInt), literal(Sarg[(0..+∞); NULL AS TRUE])),
and(isTrue(vBoolNotNull(0)), eq(vBool(0), vBool(0)),
rexBuilder.makeCall(SqlStdOperatorTable.SEARCH, literal(0), literal(Sarg[4]
had non-nullable type BOOLEAN, and it was optimized to >(CASE(true,
-1076415185, null:INTEGER), 0)
gt(case_(trueLiteral, literal(-1076415185), nullInt), literal(0)) that has
nullable type BOOLEAN
at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:39)
at org.junit.jupiter.api.Assertions.fail(Assertions.java:109)
at
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.checkUnknownAs(RexProgramFuzzyTest.java:306)
at
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.checkUnknownAsAndShrink(RexProgramFuzzyTest.java:173)
at
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.checkUnknownAs(RexProgramFuzzyTest.java:167)
at
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.generateRexAndCheckTrueFalse(RexProgramFuzzyTest.java:457)
at
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.runRexFuzzer(RexProgramFuzzyTest.java:389)
at
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.testFuzzy(RexProgramFuzzyTest.java:360)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)
at
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125)
at
org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:46)
at
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:139)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:131)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:81)
at
org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at
org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35)
at
org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at
org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
[jira] [Created] (CALCITE-4480) Make EnumerableDefaults#union a non-blocking operation
Vladimir Sitnikov created CALCITE-4480:
--
Summary: Make EnumerableDefaults#union a non-blocking operation
Key: CALCITE-4480
URL: https://issues.apache.org/jira/browse/CALCITE-4480
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.26.0
Reporter: Vladimir Sitnikov
Currently, EnumerableDefaults#union buffers all the rows before it returns the
first of them
Pros:
1) Faster iteration in case enumerable is queried multiple times
Cons:
1) The implementation does not work with infinite streams
2) Keeps memory even after iteration is finished
---
An alternative might be something like
{code:java}
public static Enumerable union(Enumerable source0,
Enumerable source1) {
Enumerable unionAll = concat(source0, source1);
return new AbstractEnumerable() {
@Override public Enumerator enumerator() {
Set set = new HashSet<>();
return EnumerableDefaults.where(unionAll, set::add).enumerator();
}
};
}
{code}
Pros:
1) Supports infinite streams
2) In theory, it could reset hashSet after iteration finishes
Cons:
1) Slower iteration in case enumerable is queried multiple times (hashSet is
rebuilt every time)
2) concat+abstractenumerable might const CPU cycles
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4470) Add optional bytecode verification with Jandex
Vladimir Sitnikov created CALCITE-4470: -- Summary: Add optional bytecode verification with Jandex Key: CALCITE-4470 URL: https://issues.apache.org/jira/browse/CALCITE-4470 Project: Calcite Issue Type: Sub-task Affects Versions: 1.26.0 Reporter: Vladimir Sitnikov Assignee: Vladimir Sitnikov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4459) Type annotations defeat org.jboss:jandex bytecode parser sometimes
Vladimir Sitnikov created CALCITE-4459:
--
Summary: Type annotations defeat org.jboss:jandex bytecode parser
sometimes
Key: CALCITE-4459
URL: https://issues.apache.org/jira/browse/CALCITE-4459
Project: Calcite
Issue Type: Improvement
Components: core, linq4j
Reporter: Vladimir Sitnikov
Assignee: Vladimir Sitnikov
For instance, the following code in
{{org.apache.calcite.linq4j.EnumerableDefaults.WrapMap}}
{code:java}
@Override public Set> entrySet() {
return new AbstractSet>() {
{code}
produces the following exception with {{org.jboss:jandex:2.0.3.Final}}
{noformat}
Caused by: java.lang.IllegalStateException: Class extends type annotation
appeared on a non class target
at org.jboss.jandex.Indexer.processTypeAnnotation(Indexer.java:403)
at org.jboss.jandex.Indexer.processTypeAnnotations(Indexer.java:380)
at org.jboss.jandex.Indexer.processAttributes(Indexer.java:314)
at org.jboss.jandex.Indexer.processMethodInfo(Indexer.java:271)
at org.jboss.jandex.Indexer.index(Indexer.java:1454)
{noformat}
Note: Calcite bytecode is perfectly valid (provided a recent javac is used),
however, jandex is used by Hibernate, Quarkus (and other projects), so it might
result in non-tirival exceptions like {{IllegalStateException: Class extends
type annotation appeared on a non class target}} while users would have
absolutely no clue which class causes failure and why.
Sample jandex issues: https://github.com/wildfly/jandex/issues/98,
https://github.com/wildfly/jandex/issues/88,
https://github.com/wildfly/jandex/issues/80, and others.
I suggest we parse Calcite-produced classes with Jandex, so we know when
incompatibility happens.
Here's the same case for pgjdbc/pgjdbc:
https://github.com/pgjdbc/pgjdbc/pull/2010
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4431) Use requireNonNull(var, "var") instead of requireNonNull(var) for better error messages
Vladimir Sitnikov created CALCITE-4431: -- Summary: Use requireNonNull(var, "var") instead of requireNonNull(var) for better error messages Key: CALCITE-4431 URL: https://issues.apache.org/jira/browse/CALCITE-4431 Project: Calcite Issue Type: Improvement Affects Versions: 1.26.0 Reporter: Vladimir Sitnikov Assignee: Vladimir Sitnikov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4422) Add MethodCanBeStatic check via ErrorProne
Vladimir Sitnikov created CALCITE-4422:
--
Summary: Add MethodCanBeStatic check via ErrorProne
Key: CALCITE-4422
URL: https://issues.apache.org/jira/browse/CALCITE-4422
Project: Calcite
Issue Type: Improvement
Affects Versions: 1.26.0
Reporter: Vladimir Sitnikov
See https://errorprone.info/bugpattern/MethodCanBeStatic
{quote}A private method that does not reference the enclosing instance can be
static{quote}
Note: the key driver is code readability. If a method is static, it is clear
that it can't access instance fields, and all it can access is provided via
arguments.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4399) Check if HepPlanner should be allowed to register non-nullable column as an equivalent to nullable
Vladimir Sitnikov created CALCITE-4399:
--
Summary: Check if HepPlanner should be allowed to register
non-nullable column as an equivalent to nullable
Key: CALCITE-4399
URL: https://issues.apache.org/jira/browse/CALCITE-4399
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.26.0
Reporter: Vladimir Sitnikov
org.apache.calcite.test.RelOptRulesTest#testReduceNullableToNotNull2
{noformat}
java.lang.AssertionError: Cannot add expression of different type to set:
set type is RecordType(INTEGER NEWCOL) NOT NULL
expression type is RecordType(INTEGER NOT NULL NEWCOL) NOT NULL
set is rel#9105:LogicalProject.(input=HepRelVertex#9104,exprs=[+($0,
CASE(=('a', 'a'), 1, null:INTEGER))])
expression is LogicalProject(NEWCOL=[+($0, 1)])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
at
org.apache.calcite.plan.RelOptUtil.verifyTypeEquivalence(RelOptUtil.java:380)
at
org.apache.calcite.plan.hep.HepRuleCall.transformTo(HepRuleCall.java:58)
at
org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:268)
at
org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:283)
at
org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:310)
at
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:333)
at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:542)
at
org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:407)
at
org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:243)
at
org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127)
at
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:202)
at
org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:189)
at
org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:122)
at
org.apache.calcite.test.RelOptTestBase.access$000(RelOptTestBase.java:64)
at
org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:303)
at
org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:282)
at
org.apache.calcite.test.RelOptRulesTest.checkReduceNullableToNotNull(RelOptRulesTest.java:2854)
at
org.apache.calcite.test.RelOptRulesTest.testReduceNullableToNotNull2(RelOptRulesTest.java:2871)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4398) RexSimplify introduces cast(nullable as nonnull)
Vladimir Sitnikov created CALCITE-4398:
--
Summary: RexSimplify introduces cast(nullable as nonnull)
Key: CALCITE-4398
URL: https://issues.apache.org/jira/browse/CALCITE-4398
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.26.0
Reporter: Vladimir Sitnikov
{noformat}
isNotNull(case_(case_(trueLiteral, isFalse(lt(vInt(1), literal(0))),
trueLiteral), trueLiteral, falseLiteral))
{noformat}
is optimized to
{noformat}IS NOT NULL(CAST(>=(?0.int1, 0)):BOOLEAN NOT NULL){noformat}
which looks strange, since it casts nullable value to a non-null type.
/cc [~kgyrtkirk]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4397) StandardConvertletTable#convertDatetimeMinus is inconsistent with MINUS_DATE return type inference
Vladimir Sitnikov created CALCITE-4397:
--
Summary: StandardConvertletTable#convertDatetimeMinus is
inconsistent with MINUS_DATE return type inference
Key: CALCITE-4397
URL: https://issues.apache.org/jira/browse/CALCITE-4397
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.26.0
Reporter: Vladimir Sitnikov
SqlNode representation has 3 parameters: {{datetime}} - {{datetime}}
{{interval_qualifier}}
The return type inference is {{ARG2_NULLABLE}} so it assumes the call has three
parameters.
In practice, {{convertDatetimeMinus}} removes the last parameter via
{{rexBuilder.makeCall(resType, op, exprs.subList(0, 2));}}, which blocks node
cloning via {{rexBuilder.makeCall(e.getOperator(), operands)}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4388) RexNode#isAlwaysFalse and isAlwaysTrue should be aligned with RexSimplify#isSafeExpression
Vladimir Sitnikov created CALCITE-4388:
--
Summary: RexNode#isAlwaysFalse and isAlwaysTrue should be aligned
with RexSimplify#isSafeExpression
Key: CALCITE-4388
URL: https://issues.apache.org/jira/browse/CALCITE-4388
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.26.0
Reporter: Vladimir Sitnikov
{{(1/0) IS NULL}} should not result in {{isAlwaysFalse}}.
Currently, {{RexSimplify}} knows that 1/0 must not be simplified, however,
{{isAlwaysTrue}} and {{isAlwaysFalse}} are still wrong.
That might result in wrong data, especially when the client code calls
{{isAlways...}}
[~kgyrtkirk], I see you contributed a lot to simplification and
{{SafeRexVisitor}}. Are you interested in improving {{isAlways...}}?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4387) Document that RexSimplify.simplifyUnknownAs could narrow or widen nullness of the given expression
Vladimir Sitnikov created CALCITE-4387:
--
Summary: Document that RexSimplify.simplifyUnknownAs could narrow
or widen nullness of the given expression
Key: CALCITE-4387
URL: https://issues.apache.org/jira/browse/CALCITE-4387
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.26.0
Environment: It is expected that {{RexSimplify.simplifyUnknownAs}}
would make the expression "simpler" (e.g. less RexNodes or something like
that), however, there's no explicit contract on the resulting expression.
For instance the following expression has non-nullable type
{noformat}
case_(isUnknown(vBool(1)), trueLiteral, vBool(1), falseLiteral, trueLiteral)
{noformat}
however, it might be simplified to the following **nullable** expression:
{noformat}
or(isUnknown(vBool(1)), not(vBool(1)))
{noformat}
Note: the expression never returns null, however, it might be not that obvious
that the simplified expression never returns null.
I suggest we add explicit note to {{simplifyUnknownAs}} method that says the
method is allowed to narrow or widen the nullness of the given expression type
{{simplifyPreservingType(RexNode, RexUnknownAs, boolean matchNullability)}}
could be used to keep nullness intact.
PS. It woud be nice to simplify {{or(isUnknown(vBool(1)), not(vBool(1)))}} to
{{isNotTrue(vBool(1))}}, however, that is another story. I guess there might be
different cases (e.g. involving more complicated cases) that would result in
nullability "widening".
Reporter: Vladimir Sitnikov
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4372) Correct specification getCumulativeCost and getNonCumulativeCost so it should always produce non-nullable value
Vladimir Sitnikov created CALCITE-4372:
--
Summary: Correct specification getCumulativeCost and
getNonCumulativeCost so it should always produce non-nullable value
Key: CALCITE-4372
URL: https://issues.apache.org/jira/browse/CALCITE-4372
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.26.0
Reporter: Vladimir Sitnikov
Assignee: Vladimir Sitnikov
Currently javadoc says {{getNonCumulativeCost}} can return {{null}}, however, a
lot of usages assume the resulting value is not null.
For instance, FLINK-11973 is relevant here.
Even though plugins can install custom {{getNonCumulativeCost}} handlers, we
should make {{get*CumulativeCost}} non-nullable, so {{RelOptPlanner.getCost}}
could be declared and implemented as non-nullable (which is the way the current
code behaves)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4371) TableMacro and TableFunction should have List rather than List parameters
Vladimir Sitnikov created CALCITE-4371:
--
Summary: TableMacro and TableFunction should have List rather than List parameters
Key: CALCITE-4371
URL: https://issues.apache.org/jira/browse/CALCITE-4371
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.26.0
Reporter: Vladimir Sitnikov
Assignee: Vladimir Sitnikov
Currently, the functions are declared as follows:
{code:java}public interface TableMacro extends Function {
/**
* Applies arguments to yield a table.
*
* @param arguments Arguments
* @return Table
*/
TranslatableTable apply(List arguments);
}
{code}
That means the method can't be called with {{List}} parameter.
It makes sense to adjust the parameter type to {{List}}, then
the call with {{List}} would succeed.
Note: this is source-incompatible change. So all the classes that implement
TableMacro and TableFunction would need to be adjusted.
Calcite codebase has <10 such classes, so I believe it should be ok.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4339) Update Gradle: 6.6 -> 6.7
Vladimir Sitnikov created CALCITE-4339: -- Summary: Update Gradle: 6.6 -> 6.7 Key: CALCITE-4339 URL: https://issues.apache.org/jira/browse/CALCITE-4339 Project: Calcite Issue Type: Improvement Components: build Affects Versions: 1.26.0 Reporter: Vladimir Sitnikov Assignee: Vladimir Sitnikov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4336) Add cancel-workflow-runs to automatically cancel GitHub stale workflows
Vladimir Sitnikov created CALCITE-4336: -- Summary: Add cancel-workflow-runs to automatically cancel GitHub stale workflows Key: CALCITE-4336 URL: https://issues.apache.org/jira/browse/CALCITE-4336 Project: Calcite Issue Type: Improvement Affects Versions: 1.26.0 Reporter: Vladimir Sitnikov GitHub Actions CI jobs keep running even in case developer pushes one more commit to the PR branch, however, in most of the cases we do not need that. We should cancel stale workflows to save resources and limit CI usage (there's overall concurrency limit for all ASF jobs in GitHub Actions CI) See https://github.com/potiuk/cancel-workflow-runs -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4332) Improve error when planning rule produces a relation with a wrong rowtype
Vladimir Sitnikov created CALCITE-4332:
--
Summary: Improve error when planning rule produces a relation with
a wrong rowtype
Key: CALCITE-4332
URL: https://issues.apache.org/jira/browse/CALCITE-4332
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.26.0
Reporter: Vladimir Sitnikov
Current coed in {{VolcanoPlanner}}:
{code:java}
assert RelOptUtil.equal(
"rel rowtype",
rel.getRowType(),
"equivRel rowtype",
equivRel.getRowType(),
Litmus.THROW);
equivRel = ensureRegistered(equivRel, null);
{code}
The suggested changes:
1) Use regular error {{IllegalArgumentException}} rather than an assertion
2) Make sure the exception message contains fine-grained information on what
went wrong. For instance: {{name: non nullable -> nullable; description:
character -> int; ...}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4329) EnumerableRules#BRIDGE_METHODS should be skipped when actual type is Object
Vladimir Sitnikov created CALCITE-4329:
--
Summary: EnumerableRules#BRIDGE_METHODS should be skipped when
actual type is Object
Key: CALCITE-4329
URL: https://issues.apache.org/jira/browse/CALCITE-4329
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.26.0
Reporter: Vladimir Sitnikov
Calcite generates bridge methods even when actual parameter types are
{{Object}}, so the class becomes invalid.
Sample:
{code:java}
Caused by: org.codehaus.janino.InternalCompilerException:
Two non-abstract methods
"public int Baz$11.compare(java.lang.Object, java.lang.Object)"
have the same parameter types, declaring type and return type
, org.apache.calcite.linq4j.JoinType.LEFT, new java.util.Comparator(){
public int compare(Object v0, Object v1) {
final int c;
c = org.apache.calcite.runtime.Utilities.compareNullsLast((Comparable)
v0, (Comparable) v1);
if (c != 0) {
return c;
}
return 0;
}
public int compare(Object o0, Object o1) {
return this.compare(o0, o1);
}
}).
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4328) Update janino compiler, and configure it to produce Java 1.8+ bytecode
Vladimir Sitnikov created CALCITE-4328: -- Summary: Update janino compiler, and configure it to produce Java 1.8+ bytecode Key: CALCITE-4328 URL: https://issues.apache.org/jira/browse/CALCITE-4328 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.26.0 Reporter: Vladimir Sitnikov Newer bytecode is needed to call static methods in interfaces (which is useful for method-only utility classes) See also https://github.com/janino-compiler/janino/issues/69 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4327) RelBuilder#scan(...) should work when used in RelRule#onMatch as call.builder()
Vladimir Sitnikov created CALCITE-4327:
--
Summary: RelBuilder#scan(...) should work when used in
RelRule#onMatch as call.builder()
Key: CALCITE-4327
URL: https://issues.apache.org/jira/browse/CALCITE-4327
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.26.0
Reporter: Vladimir Sitnikov
It turns out {{RelBuilder#relOptSchema}} is often null.
For instance,
{code:java}
public abstract class RelOptRuleCall {
public RelBuilder builder() {
return rule.relBuilderFactory.create(rel(0).getCluster(), /* schema= */
null);
}
{code}
It leads to NPE in user code.
I suggest either populate {{relOptSchema}} with root schema by default or
remove the field to avoid confusion for those who would try {{scan(...)}}
method.
It might make sense to add RelOptSchema parameter to {{scan(Iterable)}}
method as well.
Sample use case:
{code:java}
@Override
public void onMatch(RelOptRuleCall call) {
List indexName = ...;
RelBuilder relBuilder = call.builder()
.scan(indexName);
{code}
{noformat}
Caused by: java.lang.NullPointerException
at org.apache.calcite.tools.RelBuilder.scan(RelBuilder.java:1094)
at
com.github.vlsi.mat.calcite.schema.objects.InstanceAccessByClassIdRule.onMatch(InstanceAccessByClassIdRule.java:44)
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:229)
... 75 more
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4326) NPE possible win EnumerableWindow when agg.call.name is null
Vladimir Sitnikov created CALCITE-4326:
--
Summary: NPE possible win EnumerableWindow when agg.call.name is
null
Key: CALCITE-4326
URL: https://issues.apache.org/jira/browse/CALCITE-4326
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.26.0
Reporter: Vladimir Sitnikov
{code:java}
final RelDataTypeFactory.Builder typeBuilder = typeFactory.builder();
typeBuilder.addAll(inputPhysType.getRowType().getFieldList());
for (AggImpState agg : aggs) {
typeBuilder.add(agg.call.name, agg.call.type);
}
{code}
AggregateCall#name is nullable
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4320) JdbcRules JOIN_FACTORY, AGGREGATE_FACTORY, SET_OP_FACTORY, ... create relations with wrong convention
Vladimir Sitnikov created CALCITE-4320:
--
Summary: JdbcRules JOIN_FACTORY, AGGREGATE_FACTORY,
SET_OP_FACTORY, ... create relations with wrong convention
Key: CALCITE-4320
URL: https://issues.apache.org/jira/browse/CALCITE-4320
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.26.0
Reporter: Vladimir Sitnikov
{code:java}
static final RelFactories.JoinFactory JOIN_FACTORY =
(left, right, hints, condition, variablesSet, joinType, semiJoinDone) -> {
final RelOptCluster cluster = left.getCluster();
final RelTraitSet traitSet = cluster.traitSetOf(left.getConvention());
try {
return new JdbcJoin(cluster, traitSet, left, right, condition,
variablesSet, joinType);
{code}
In practice, JdbcJoin should use JdbcConvention rather than
{{cluster.traitSetOf(left.getConvention());}}
I wonder if RelFactories in JdbcRules are dead code.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4319) NPE in TopDownRuleDriver#convert when convention is not enabled
Vladimir Sitnikov created CALCITE-4319:
--
Summary: NPE in TopDownRuleDriver#convert when convention is not
enabled
Key: CALCITE-4319
URL: https://issues.apache.org/jira/browse/CALCITE-4319
Project: Calcite
Issue Type: Sub-task
Components: core
Reporter: Vladimir Sitnikov
{code:java}VolcanoRuleMatch match = ruleQueue.popMatch(
Pair.of(rel,
m -> m.getRule() instanceof ConverterRule
&&
m.getRule().getOutTrait().satisfies(group.getTraitSet().getConvention(;
{code}
satisfies(null) fails with NPE
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4316) Avoid NPE when nulls are used in divide call
Vladimir Sitnikov created CALCITE-4316: -- Summary: Avoid NPE when nulls are used in divide call Key: CALCITE-4316 URL: https://issues.apache.org/jira/browse/CALCITE-4316 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.26.0 Reporter: Vladimir Sitnikov Assignee: Vladimir Sitnikov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4314) Enable Error Prone checking
Vladimir Sitnikov created CALCITE-4314: -- Summary: Enable Error Prone checking Key: CALCITE-4314 URL: https://issues.apache.org/jira/browse/CALCITE-4314 Project: Calcite Issue Type: Improvement Components: build Reporter: Vladimir Sitnikov Assignee: Vladimir Sitnikov Fix For: avatica-1.14.0 [http://errorprone.info/] provides the ability to do static checks like Spotbugs (CALCITE-1006). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4310) Add API backward compatibility verification
Vladimir Sitnikov created CALCITE-4310:
--
Summary: Add API backward compatibility verification
Key: CALCITE-4310
URL: https://issues.apache.org/jira/browse/CALCITE-4310
Project: Calcite
Issue Type: Improvement
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
One of the key questions when preparing a release is "what are the breaking
changes?", and it would be nice if we could document the changes and minimize
them.
Here are the tools that might help:
* japicmp: https://github.com/melix/japicmp-gradle-plugin (integrated with
Gradle, enables to configure extra filters so we could exclude
{{@API(internal)}} from public apis)
* https://abi-laboratory.pro/java/tracker/ : (no idea how it compares with
japicmp)
* Kotlin-specific: https://github.com/Kotlin/binary-compatibility-validator
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4309) Clarify org.apache.calcite.sql.type.SqlTypeName#getFamily nullability
Vladimir Sitnikov created CALCITE-4309:
--
Summary: Clarify org.apache.calcite.sql.type.SqlTypeName#getFamily
nullability
Key: CALCITE-4309
URL: https://issues.apache.org/jira/browse/CALCITE-4309
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
Currently {{SqlTypeFamily SqlTypeName#getFamily()}} is nullable, and in
practice, only the following enum members have null family:
{code:java}
SYMBOL(PrecScale.NO_NO, true, Types.OTHER, null),
DISTINCT(PrecScale.NO_NO, false, Types.DISTINCT, null),
STRUCTURED(PrecScale.NO_NO, false, Types.STRUCT, null),
ROW(PrecScale.NO_NO, false, Types.STRUCT, null),
OTHER(PrecScale.NO_NO, false, Types.OTHER, null),
{code}
Should we replace {{null}} family with a non-null value?
Should we keep it as is and use `requireNonNull` when use-site expects a
non-null value?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4308) JointNode in Iterpreter might fail with NPE for FULL join
Vladimir Sitnikov created CALCITE-4308:
--
Summary: JointNode in Iterpreter might fail with NPE for FULL join
Key: CALCITE-4308
URL: https://issues.apache.org/jira/browse/CALCITE-4308
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Environment:
https://github.com/apache/calcite/blob/64a0ca71038da14f0f883f169aa95df00d909e06/core/src/main/java/org/apache/calcite/interpreter/JoinNode.java#L85
{code:java}
if (rel.getJoinType() == JoinRelType.FULL) {
// send un-match rows for full join on right source
List empty = new ArrayList<>();
for (Row row: innerRows) { // <-- NPE here
if (matchRowSet.contains(row)) {
continue;
}
doSend(row, empty, JoinRelType.RIGHT);
}
}
{code}
Reporter: Vladimir Sitnikov
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4307) Compile code with parameter names (add -parameters to javac)
Vladimir Sitnikov created CALCITE-4307: -- Summary: Compile code with parameter names (add -parameters to javac) Key: CALCITE-4307 URL: https://issues.apache.org/jira/browse/CALCITE-4307 Project: Calcite Issue Type: Improvement Affects Versions: 1.25.0 Reporter: Vladimir Sitnikov Currently the code is compiled without parameter names, so Jackson can't figure out JsonProperty names from the constructor parameter names. Calcite is Java 1.8+, so we could add -parameters, and start using it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4306) ViewTableMacro#schemaPath is always initialized to non-null value, however, it is treated like nullable
Vladimir Sitnikov created CALCITE-4306:
--
Summary: ViewTableMacro#schemaPath is always initialized to
non-null value, however, it is treated like nullable
Key: CALCITE-4306
URL: https://issues.apache.org/jira/browse/CALCITE-4306
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
JavaDoc says:
{code:java}
/** Typically null. If specified, overrides the path of the schema as the
* context for validating {@code viewSql}. */
protected final @Nullable List schemaPath;
{code}
However, all the current instantiations use non-null value for schemaPath.
Should the field be converted to non-nullable?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4304) Make org.apache.calcite.jdbc.JavaTypeFactoryImpl#getJavaClass non-nullable
Vladimir Sitnikov created CALCITE-4304:
--
Summary: Make
org.apache.calcite.jdbc.JavaTypeFactoryImpl#getJavaClass non-nullable
Key: CALCITE-4304
URL: https://issues.apache.org/jira/browse/CALCITE-4304
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
Assignee: Vladimir Sitnikov
The current implementation returns {{null}} as the last resort, however, it
looks like none of the use-sites would handle that properly.
I suggest we throw IllegalArgumentException if {{getJavaClass}} can't deal with
given argument.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4303) Refactor org.apache.calcite.rel.RelInput to provide null-safe getters
Vladimir Sitnikov created CALCITE-4303:
--
Summary: Refactor org.apache.calcite.rel.RelInput to provide
null-safe getters
Key: CALCITE-4303
URL: https://issues.apache.org/jira/browse/CALCITE-4303
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
There are several methods which might return null, however, they are used a lot
as "never null" ones:
{code:java}
> @Nullable E getEnum(String tag, Class enumClass);
@Nullable List getExpressionList(String tag);
@Nullable List getStringList(String tag);
@Nullable List getIntegerList(String tag);
@Nullable List> getIntegerListList(String tag);
{code}
It would be nice if the methods returned non-nullable values.
The suggested API is
{code:java}
> E getEnum(String tag, Class enumClass);
List getExpressionList(String tag);
List getStringList(String tag);
List getIntegerList(String tag);
List> getIntegerListList(String tag);
> @Nullable E getEnumOrNull(String tag, Class enumClass);
@Nullable List getExpressionListOrNull(String tag);
@Nullable List getStringListOrNull(String tag);
@Nullable List getIntegerListOrNull(String tag);
@Nullable List> getIntegerListListOrNull(String tag);
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4288) Create SqlTypeUtil#deriveType(SqlCallBinding) to make type computation simpler
Vladimir Sitnikov created CALCITE-4288:
--
Summary: Create SqlTypeUtil#deriveType(SqlCallBinding) to make
type computation simpler
Key: CALCITE-4288
URL: https://issues.apache.org/jira/browse/CALCITE-4288
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
Assignee: Vladimir Sitnikov
The following pattern repeats quite often, so it is worth extracting to a
separate method.
{code:java}
binding.getValidator().deriveType(binding.getScope(), binding.getCall())
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4286) Remove NonNull annotations
Vladimir Sitnikov created CALCITE-4286:
--
Summary: Remove NonNull annotations
Key: CALCITE-4286
URL: https://issues.apache.org/jira/browse/CALCITE-4286
Project: Calcite
Issue Type: Sub-task
Components: core, druid-adapter
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
Assignee: Vladimir Sitnikov
{{NonNull}} is the default convention, so there's no need to keep them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4285) Replace jsr305 dependency with checkerframework and errorprone_annotations
Vladimir Sitnikov created CALCITE-4285:
--
Summary: Replace jsr305 dependency with checkerframework and
errorprone_annotations
Key: CALCITE-4285
URL: https://issues.apache.org/jira/browse/CALCITE-4285
Project: Calcite
Issue Type: Sub-task
Components: core, druid-adapter
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
Assignee: Vladimir Sitnikov
jsr305 does not support type annotations, so it can't be used like
{{List<@Nullable String>}}, and it results in duplicate {{@Nullable}}
annotations on the classpath (checkerframework and jsr305)
The solution is to use {{checkerframework}} for nullability, and {{errorprone}}
annotations for dataflow-like checks (e.g. {{@CheckReturnValue}})
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4284) ImmutableBeans: make reference properties non-nullable by default
Vladimir Sitnikov created CALCITE-4284:
--
Summary: ImmutableBeans: make reference properties non-nullable by
default
Key: CALCITE-4284
URL: https://issues.apache.org/jira/browse/CALCITE-4284
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
The rule of thumb is to avoid {{null}}, and it would be better to make
properties non-nullable by default, and use annotations to permit nullable
types.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4280) Replace Guava's Lists.transform with Util.transform
Vladimir Sitnikov created CALCITE-4280: -- Summary: Replace Guava's Lists.transform with Util.transform Key: CALCITE-4280 URL: https://issues.apache.org/jira/browse/CALCITE-4280 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.25.0 Environment: Lists.transform has wrong generic signature, and it relies on Guava's Function types. We should stick to a single method, so I suggest we go with Util.transform, and we should probably add Lists.transform to forbidden-apis list. Reporter: Vladimir Sitnikov Assignee: Vladimir Sitnikov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4268) org.apache.calcite.model.ModelHandler#checkRequiredAttributes should use Objects.requireNonNull
Vladimir Sitnikov created CALCITE-4268:
--
Summary:
org.apache.calcite.model.ModelHandler#checkRequiredAttributes should use
Objects.requireNonNull
Key: CALCITE-4268
URL: https://issues.apache.org/jira/browse/CALCITE-4268
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
{{ModelHandler#checkRequiredAttributes}} uses reflection to ensure fields are
non-null, however, {{Objects.requireNonNull}} or something like that would be
easier to follow.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4263) Clarify RelNode#estimateRowCount behavior with regards to nulls
Vladimir Sitnikov created CALCITE-4263:
--
Summary: Clarify RelNode#estimateRowCount behavior with regards to
nulls
Key: CALCITE-4263
URL: https://issues.apache.org/jira/browse/CALCITE-4263
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
{{RelNode#estimateRowCount}} implementation must return non-nullable value, and
it often uses metadata which might return {{null}}
For instance,
{code:java}
class Correlate {
@Override public double estimateRowCount(RelMetadataQuery mq) {
double leftRowCount = mq.getRowCount(left); // <-- NPE here
switch (joinType) {
case SEMI:
case ANTI:
return leftRowCount;
default:
return leftRowCount * mq.getRowCount(right); // <-- NPE here
}
}
{code}
There should be documentation on the desired behavior in case metadata returns
"unknown", and the code needs adjustment as lot of the implementation perform
blind unboxing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4253) RelOptUtil#findAllTables should probably use RelMetadataQuery#getTableReferences
Vladimir Sitnikov created CALCITE-4253:
--
Summary: RelOptUtil#findAllTables should probably use
RelMetadataQuery#getTableReferences
Key: CALCITE-4253
URL: https://issues.apache.org/jira/browse/CALCITE-4253
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
It looks like both methods do exactly the same thing, so it would probably make
sense to divert {{findAllTables}} to {{RelMetadataQuery#getTableReferences}}
If methods are different, it would make sense to add the relevant documentation
and cross-references.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4252) Add build validation for improper dependency declaration
Vladimir Sitnikov created CALCITE-4252:
--
Summary: Add build validation for improper dependency declaration
Key: CALCITE-4252
URL: https://issues.apache.org/jira/browse/CALCITE-4252
Project: Calcite
Issue Type: Improvement
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
We want to avoid declaring unused dependencies, and we want to declare all the
used dependencies (we do not want to rely on transitive dependencies).
The relevant configuration was present in Maven: CALCITE-658
Now it's time to re-add it thanks to
https://github.com/autonomousapps/dependency-analysis-android-gradle-plugin
Note: even though there'es Android in the name, the plugin is not tied to
Android.
Sample output:
{noformat}
> Task :core:aggregateAdvice
Transitively used dependencies that should be declared directly as indicated:
- implementation("log4j:log4j:1.2.17")
- implementation("org.apache.commons:commons-pool2:2.6.1")
Existing dependencies which should be modified to be as indicated:
- api("org.slf4j:slf4j-api:1.7.25") (was implementation)
- implementation("org.apiguardian:apiguardian-api:1.1.0") (was api)
- api("com.google.guava:guava:19.0") (was implementation)
- implementation("com.fasterxml.jackson.core:jackson-annotations:2.10.0") (was
api)
- testImplementation("org.apache.commons:commons-dbcp2:2.6.0") (was
implementation)
- api("com.esri.geometry:esri-geometry-api:2.2.0") (was implementation)
> Task :linq4j:aggregateAdvice
Existing dependencies which should be modified to be as indicated:
- implementation("org.apiguardian:apiguardian-api:1.1.0") (was api)
> Task :example:function:aggregateAdvice
Unused dependencies which should be removed:
- implementation("com.google.guava:guava:19.0")
> Task :file:aggregateAdvice
Transitively used dependencies that should be declared directly as indicated:
- implementation("com.fasterxml.jackson.core:jackson-databind:2.10.0")
- implementation("com.fasterxml.jackson.core:jackson-core:2.10.0")
> Task :babel:aggregateAdvice
Existing dependencies which should be modified to be as indicated:
- api("org.apache.calcite.avatica:avatica-core:1.17.0") (was implementation)
Plugins that should be removed:
- org.jetbrains.kotlin.jvm, because this project has both java-library and
org.jetbrains.kotlin.jvm applied, which is redundant. You can remove
org.jetbrains.kotlin.jvm
> Task :kafka:aggregateAdvice
Existing dependencies which should be modified to be as indicated:
- api("org.apache.kafka:kafka-clients:2.1.1") (was implementation)
> Task :cassandra:aggregateAdvice
Existing dependencies which should be modified to be as indicated:
- api("com.google.guava:guava:19.0") (was implementation)
- api("org.slf4j:slf4j-api:1.7.25") (was implementation)
- api("com.datastax.cassandra:cassandra-driver-core:3.6.0") (was implementation)
> Task :druid:aggregateAdvice
Existing dependencies which should be modified to be as indicated:
- api("com.fasterxml.jackson.core:jackson-core:2.10.0") (was implementation)
- api("joda-time:joda-time:2.8.1") (was implementation)
- api("org.slf4j:slf4j-api:1.7.25") (was implementation)
- api("org.apache.calcite.avatica:avatica-core:1.17.0") (was implementation)
> Task :elasticsearch:aggregateAdvice
Transitively used dependencies that should be declared directly as indicated:
- implementation("org.apache.httpcomponents:httpasyncclient:4.1.4")
Existing dependencies which should be modified to be as indicated:
- api("com.fasterxml.jackson.core:jackson-databind:2.10.0") (was implementation)
- implementation("com.fasterxml.jackson.core:jackson-annotations:2.10.0") (was
api)
- api("org.elasticsearch.client:elasticsearch-rest-client:7.0.1") (was
implementation)
> Task :geode:aggregateAdvice
Transitively used dependencies that should be declared directly as indicated:
- implementation("com.fasterxml.jackson.core:jackson-databind:2.10.0")
Existing dependencies which should be modified to be as indicated:
- api("org.apache.geode:geode-core:1.10.0") (was implementation)
- api("org.slf4j:slf4j-api:1.7.25") (was implementation)
> Task :mongodb:aggregateAdvice
Existing dependencies which should be modified to be as indicated:
- api("com.google.guava:guava:19.0") (was implementation)
- api("org.slf4j:slf4j-api:1.7.25") (was implementation)
> Task :redis:aggregateAdvice
Unused dependencies which should be removed:
- implementation("com.google.code.findbugs:jsr305:3.0.1")
- implementation("joda-time:joda-time:2.8.1")
Existing dependencies which should be modified to be as indicated:
- testImplementation("com.github.kstyrc:embedded-redis:0.6") (was
implementation)
- api("redis.clients:jedis:2.9.0") (was implementation)
> Task :splunk:aggregateAdvice
Existing dependencies which should be modified to be as indicated:
- api("com.google.guava:guava:19.0") (was implementation)
- api("org.slf4j:slf4j-api:1.7.25") (was implementation)
-
[jira] [Created] (CALCITE-4235) SqlValidatorImpl#validateGroupItem and validateGroupingSets are not using AggregatingSelectScope parameter
Vladimir Sitnikov created CALCITE-4235: -- Summary: SqlValidatorImpl#validateGroupItem and validateGroupingSets are not using AggregatingSelectScope parameter Key: CALCITE-4235 URL: https://issues.apache.org/jira/browse/CALCITE-4235 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.25.0 Reporter: Vladimir Sitnikov The methods are passing AggregatingSelectScope aggregatingScope parameter to each other, however, none of them is using the parameter. So the parameter might need to be removed or it might mean the parameter needs to be used for some validation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4234) org.apache.calcite.tools.RelBuilder.AggCallImpl#aggregateCall() should not throw NPE
Vladimir Sitnikov created CALCITE-4234:
--
Summary:
org.apache.calcite.tools.RelBuilder.AggCallImpl#aggregateCall() should not
throw NPE
Key: CALCITE-4234
URL: https://issues.apache.org/jira/browse/CALCITE-4234
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
Assignee: Julian Hyde
CALCITE-4220 introduced the following method:
{code:java}
@Override public AggregateCall aggregateCall() {
return AggregateCall.create(aggFunction, distinct, approximate,
ignoreNulls, ImmutableList.of(), -1, /* collation: */ null, /* type:
*/ null, alias);
}
{code}
In practice {{AggregateCall}} constructor has {{null}} checks, so
{{aggregateCall}} would always throw NPE:
{code:java}
this.type = Objects.requireNonNull(type);
this.collation = Objects.requireNonNull(collation);
{code}
[~julianhyde], would you please check if that behavior is expected?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4231) org.apache.calcite.rex.RexProgram.Marshaller might need overriding visitSubQuery / visitTableInputRef / visitPatternFieldRef
Vladimir Sitnikov created CALCITE-4231: -- Summary: org.apache.calcite.rex.RexProgram.Marshaller might need overriding visitSubQuery / visitTableInputRef / visitPatternFieldRef Key: CALCITE-4231 URL: https://issues.apache.org/jira/browse/CALCITE-4231 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.25.0 Reporter: Vladimir Sitnikov Marshaller is used in org.apache.calcite.rex.RexProgram#gatherExpr only, so it might be dead code. We might want to remove the dead code, or fix it, or leave as is -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4229) Add Util.throwAsRuntime and Util.causeOrSelf to simplify exception re-throwing
Vladimir Sitnikov created CALCITE-4229: -- Summary: Add Util.throwAsRuntime and Util.causeOrSelf to simplify exception re-throwing Key: CALCITE-4229 URL: https://issues.apache.org/jira/browse/CALCITE-4229 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.25.0 Reporter: Vladimir Sitnikov Assignee: Vladimir Sitnikov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4228) FlatLists.Flat6List#append would throw NPE if there are null elements in the list
Vladimir Sitnikov created CALCITE-4228:
--
Summary: FlatLists.Flat6List#append would throw NPE if there are
null elements in the list
Key: CALCITE-4228
URL: https://issues.apache.org/jira/browse/CALCITE-4228
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
The implementation should use {{ImmutableNullableList}} rather than
{{ImmutableList}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4227) ImmutableIntList#toArray(Integer[]) should support arguments larger than the collection itself
Vladimir Sitnikov created CALCITE-4227:
--
Summary: ImmutableIntList#toArray(Integer[]) should support
arguments larger than the collection itself
Key: CALCITE-4227
URL: https://issues.apache.org/jira/browse/CALCITE-4227
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Environment: {code:java}
assertThat(
Arrays.toString(ImmutableIntList.of(1).toArray(new Integer[]{5,6,7})),
is("[1, null, 7]")
);
{code}
{noformat}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.calcite.util.ImmutableIntList.toArray(ImmutableIntList.java:161)
[0;1mat
org.apache.calcite.util.UtilTest.testImmutableIntList(UtilTest.java:1127)
{noformat}
Reporter: Vladimir Sitnikov
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4226) Mappings#asList might return null for negative ordinals which would result in later NPEs
Vladimir Sitnikov created CALCITE-4226:
--
Summary: Mappings#asList might return null for negative ordinals
which would result in later NPEs
Key: CALCITE-4226
URL: https://issues.apache.org/jira/browse/CALCITE-4226
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
It looks like none of the uses for Mappings#asList is ready to handle nulls.
So far there are 6 usages of the method, however, it is puzzling if NPE is
expected behavior there.
I'm inclined to create two variations of the method:
1) {{List<@Nullable Integer> asList(...)}} -- keep the current method current
one
2) {{List<@NonNull Integer> asList(...)}} -- throw exceptions on invalid
mappings
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4222) Clarify if null elements are allowed in RelTraitSet
Vladimir Sitnikov created CALCITE-4222:
--
Summary: Clarify if null elements are allowed in RelTraitSet
Key: CALCITE-4222
URL: https://issues.apache.org/jira/browse/CALCITE-4222
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
Currently, one-half of the methods in {{RelTraitSet}} fails with NPE, and
another half does have some null-checks. Then there are {{getDefault()}} and
{{isDefault()}}
AFAIK the only way to create a traitset with null element currently is to
implement {{org.apache.calcite.plan.RelTraitDef#getDefault}} to return {{null}}
(which never happens in Calcite code).
However, such a traitset would likely fail with NPE in
{{org.apache.calcite.plan.RelTraitSet#satisfies}} quite fast.
{{RelCollationTraitDef}} and {{RelDistributionTraitDef}} have non-null default
value which might interfere with collation and distribution metadata. For
instance, {{distribution}} metadata might return null, and then it would fail
with NPE when replacing distribution trait.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4219) Clarify SqlCall#getOperandList() nullability
Vladimir Sitnikov created CALCITE-4219:
--
Summary: Clarify SqlCall#getOperandList() nullability
Key: CALCITE-4219
URL: https://issues.apache.org/jira/browse/CALCITE-4219
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
{{getOperandList()}} is implemented by a number of different {{SqlCall}}
sub-classes, and the list often includes null values.
The implementation is typically {{return ImmutableNullableList.of(...}}
However, {{getOperandList()}} is used a lot, and the code assumes that the
resulting values are non-null. The same happens for {{SqlCall#operand(int)}}.
The workaround is to declare {{getOperandList()}} and {{operand(int)}} to
return non-nullable values (even though nulls are possible), and we could add
suppressions at the side which returns {{ImmutableNullableList}}.
The solution might be behind the lines of adding
{{nonNullableOperand(int)}}-like methods that would return non-nullable values
and throw meaningful errors in case the value turns out to be null.
Another option might be to ensure all the fields are non-nullable (e.g. create
dummy SqlNode for empty values).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4218) Clarify RelMetadataQuery#RelDistribution nullability
Vladimir Sitnikov created CALCITE-4218:
--
Summary: Clarify RelMetadataQuery#RelDistribution nullability
Key: CALCITE-4218
URL: https://issues.apache.org/jira/browse/CALCITE-4218
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
It looks like {{null}} is the same as {{RelDistributions.ANY}}, so we might
want to make {{RelDistribution}} non-nullable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4217) RelCrossType#isStruct should be true rather than false
Vladimir Sitnikov created CALCITE-4217:
--
Summary: RelCrossType#isStruct should be true rather than false
Key: CALCITE-4217
URL: https://issues.apache.org/jira/browse/CALCITE-4217
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
Currently RelCrossType#isStruct returns false even though the default
implementation in {{org.apache.calcite.rel.type.RelDataTypeImpl#isStruct}} is
{{return fieldList != null}}
That results in {{getFieldList}} throwing assertion error (because it asserts
{{isStruct}}), however RelCrossType#getFieldNames works OK because it asserts
{{fieldList != null}}
I think it should be OK if we remove {{isStruct() { return false; } }} override
and use {{isStruct}} assertion in both {{getFieldList}} and {{getFieldNames}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4216) Ensure org.apache.calcite.rel.type.RelDataType#getSqlTypeName is non-null
Vladimir Sitnikov created CALCITE-4216:
--
Summary: Ensure
org.apache.calcite.rel.type.RelDataType#getSqlTypeName is non-null
Key: CALCITE-4216
URL: https://issues.apache.org/jira/browse/CALCITE-4216
Project: Calcite
Issue Type: Sub-task
Components: core
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
Existing code treats RelDataType#getSqlTypeName as non-nullable, and
org.apache.calcite.rel.type.RelDataTypeImpl#getSqlTypeName implements the
method as {{return null}}, howeve, RelDataTypeImpl is abstract.
We should make it explicit that getSqlTypeName should be non-null.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4215) Ensure org.apache.calcite.schema.Statistic uses null vs emptyList appropriately
Vladimir Sitnikov created CALCITE-4215: -- Summary: Ensure org.apache.calcite.schema.Statistic uses null vs emptyList appropriately Key: CALCITE-4215 URL: https://issues.apache.org/jira/browse/CALCITE-4215 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.25.0 Reporter: Vladimir Sitnikov null: statistic is *not* *known* emptyList: statistic is *known*, and the value is *empty* (e.g. no unique keys in the table) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4214) Make RelDataType.getSqlTypeName non-nullable
Vladimir Sitnikov created CALCITE-4214: -- Summary: Make RelDataType.getSqlTypeName non-nullable Key: CALCITE-4214 URL: https://issues.apache.org/jira/browse/CALCITE-4214 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.25.0 Reporter: Vladimir Sitnikov The code treated RelDataType.getSqlTypeName as non-nullable, so we need to update the javadoc and deal with abstract implementation that returns null. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4199) Add nullability annotations to the methods and fields, ensure consistency with checkerframework
Vladimir Sitnikov created CALCITE-4199:
--
Summary: Add nullability annotations to the methods and fields,
ensure consistency with checkerframework
Key: CALCITE-4199
URL: https://issues.apache.org/jira/browse/CALCITE-4199
Project: Calcite
Issue Type: New Feature
Affects Versions: 1.25.0
Reporter: Vladimir Sitnikov
The current codebase uses jsr305 javax.annotation.Nullable / NonNull which
helps to catch bugs while developing Calcite and libraries.
Unfortunately, the current annotation is not enforced, and it lacks support for
generics.
jsr305.jar is probably abandoned (see
https://github.com/google/guava/issues/2960), so we should probably migrate to
something else.
https://checkerframework.org/ is a solid framework for machine verification
which is tailored to Java.
The releases are quite frequent:
https://github.com/typetools/checker-framework/releases
Their annotations are recognized by major IDEs.
So I see the following options:
a) Leave the code as is
b) Annotate (gradually?) the code with checkerframework annotations
c) Migrate (gradually?) the code to Kotlin
I've created a PR to verify which changes would be needed, verify if CI is able
to type check in a reasonable time, and so on:
https://github.com/apache/calcite/pull/1929
My findings regarding checkerframework so far are:
0) It does detect NPEs (which were hidden in the current code), and it does
detect initialized files in the constructors
1) Checkerframework takes ~2 minutes for linq4j, and 13+min (unknown yet, 13m
produces 100 errors and stops) for core
2) "non-nullable by default" is quite close to the current Calcite conventions.
3) There are cases when javadoc comments says "or null", however, the code
reads much easier if {{@Nullable}} nullable appears in the signature
4) If a third-party library supplies invalid type annotations, there's a way to
fix that. For instance, Guava's Function is annotated as "always nullable", and
we can overrule that (so the nullability is taken from generic signature rather
than "always nullable"). The override files are placed to
src/main/config/checkerframework
5) Generic-heavy code might be challenging (they are always like that),
however, in the most obscure cases there's always a way to suppress the warning
6) I've filed a Gradle improvement so it schedules recently modified files
first for the compilation: https://github.com/gradle/gradle/issues/14332
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4187) GeodeBookstoreTest > initializationError: CacheClosedException: A cache has not yet been created
Vladimir Sitnikov created CALCITE-4187:
--
Summary: GeodeBookstoreTest > initializationError:
CacheClosedException: A cache has not yet been created
Key: CALCITE-4187
URL: https://issues.apache.org/jira/browse/CALCITE-4187
Project: Calcite
Issue Type: Task
Components: geode-adapter
Affects Versions: 1.24.0
Reporter: Vladimir Sitnikov
https://github.com/apache/calcite/runs/1010002016#step:4:916
{noformat}
FAILURE 0.0sec, org.apache.calcite.adapter.geode.rel.GeodeBookstoreTest >
initializationError
java.lang.IllegalStateException: Expected state not responding but got
online
at
com.google.common.base.Preconditions.checkState(Preconditions.java:199)
at
org.apache.calcite.adapter.geode.rel.GeodeEmbeddedPolicy.requireStatus(GeodeEmbeddedPolicy.java:94)
at
org.apache.calcite.adapter.geode.rel.GeodeEmbeddedPolicy.beforeAll(GeodeEmbeddedPolicy.java:52)
at
org.apache.calcite.adapter.geode.rel.GeodeEmbeddedPolicy$RefCountPolicy.beforeAll(GeodeEmbeddedPolicy.java:138)
at
org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeBeforeAllCallbacks$7(ClassBasedTestDescriptor.java:355)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeBeforeAllCallbacks(ClassBasedTestDescriptor.java:355)
at
org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:189)
at
org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:77)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:132)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at
org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
Suppressed: org.apache.geode.cache.CacheClosedException: A cache has
not yet been created.
at
org.apache.geode.internal.cache.CacheFactoryStatics.getAnyInstance(CacheFactoryStatics.java:87)
at
org.apache.geode.cache.CacheFactory.getAnyInstance(CacheFactory.java:396)
at
org.apache.calcite.adapter.geode.rel.GeodeEmbeddedPolicy.afterAll(GeodeEmbeddedPolicy.java:58)
at
org.apache.calcite.adapter.geode.rel.GeodeEmbeddedPolicy$RefCountPolicy.afterAll(GeodeEmbeddedPolicy.java:145)
at
org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeAfterAllCallbacks$13(ClassBasedTestDescriptor.java:421)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeAfterAllCallbacks$14(ClassBasedTestDescriptor.java:421)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at
org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeAfterAllCallbacks(ClassBasedTestDescriptor.java:421)
at
org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.after(ClassBasedTestDescriptor.java:213)
at
org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.after(ClassBasedTestDescriptor.java:77)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:145)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:145)
... 5 more
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4185) Remove dependency between checkstyle and compilation tasks
Vladimir Sitnikov created CALCITE-4185: -- Summary: Remove dependency between checkstyle and compilation tasks Key: CALCITE-4185 URL: https://issues.apache.org/jira/browse/CALCITE-4185 Project: Calcite Issue Type: Task Reporter: Vladimir Sitnikov Assignee: Vladimir Sitnikov Checkstyle 8.26+ does not really use the compiled classes, so the dependency between checkstyle and compileJava could be removed. It would improve build times, and it would improve checkstyle cacheability (e.g. no need to re-execute checkstyleTest if test classes were not modified) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4184) Update Gradle: 6.3 -> 6.6
Vladimir Sitnikov created CALCITE-4184: -- Summary: Update Gradle: 6.3 -> 6.6 Key: CALCITE-4184 URL: https://issues.apache.org/jira/browse/CALCITE-4184 Project: Calcite Issue Type: Task Affects Versions: 1.24.0 Reporter: Vladimir Sitnikov Assignee: Vladimir Sitnikov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4168) Configure Gradle Remote Build Cache to speedup builds
Vladimir Sitnikov created CALCITE-4168: -- Summary: Configure Gradle Remote Build Cache to speedup builds Key: CALCITE-4168 URL: https://issues.apache.org/jira/browse/CALCITE-4168 Project: Calcite Issue Type: Task Affects Versions: 1.24.0 Reporter: Vladimir Sitnikov Assignee: Vladimir Sitnikov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4141) Make checkstyle tasks relocatable to support Gradle build cache
Vladimir Sitnikov created CALCITE-4141:
--
Summary: Make checkstyle tasks relocatable to support Gradle build
cache
Key: CALCITE-4141
URL: https://issues.apache.org/jira/browse/CALCITE-4141
Project: Calcite
Issue Type: Task
Affects Versions: 1.23.0, avatica-1.17.0
Reporter: Vladimir Sitnikov
Checkstyle configuration uses {{base_dir=rootDir.toString()}} property which
defeats caching as different workstations tend to have different absolute path
to the Calcite checkout.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4140) Configure Gradle Remote Build Cache to speedup builds
Vladimir Sitnikov created CALCITE-4140:
--
Summary: Configure Gradle Remote Build Cache to speedup builds
Key: CALCITE-4140
URL: https://issues.apache.org/jira/browse/CALCITE-4140
Project: Calcite
Issue Type: Task
Affects Versions: 1.23.0, avatica-1.17.0
Reporter: Vladimir Sitnikov
Gradle build cache enables to reuse build artifacts across different build
executions: https://docs.gradle.org/current/userguide/build_cache.html
For instance, if linq4j code is not modified, its `compileJava` and `javadoc`
results can be reused in the other builds (e.g. PR builds).
{{test}} tasks should probably not be cached as the tests often include
integrations with databases, so we should execute them from scratch.
Note: the cache is sensitive to Gradle version, Java version, and extra files
in the workspace, so the common practice is to seed the cache only from CI jobs.
There's ticket for ASF build cache:
https://issues.apache.org/jira/browse/INFRA-20516, however, it is likely we
won't be able to use it in GitHub Actions :-/
An alternative option is to setup AWS S3-backed cache via
https://github.com/burrunan/gradle-s3-build-cache
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3786) Add Digest (HashStrategy?) interface to enable efficient hashCode/equals for RexNode, RelNode
Vladimir Sitnikov created CALCITE-3786:
--
Summary: Add Digest (HashStrategy?) interface to enable efficient
hashCode/equals for RexNode, RelNode
Key: CALCITE-3786
URL: https://issues.apache.org/jira/browse/CALCITE-3786
Project: Calcite
Issue Type: New Feature
Components: core
Affects Versions: 1.21.0
Reporter: Vladimir Sitnikov
Current digests for RexNode, RelNode, RelType, and similar cases use String
concatenation.
It is easy to implement, however, it has drawbacks:
1) String objects cannot be reused. For instance, RexCall has operands,
however, the digest is duplicated. It causes extra memory use, and extra CPU
for string copying
2) There's no way to have multiple #toString() methods. RelType might need
multiple digests: "including field names", "excluding field names".
Suggested resolution might be behind the lines of
{code:java}
class Digest { // immutable
final int hashCode; // speedup hashCode and equals
final Object[] contents; // The values are either other Digest objects or
Strings
}
{code}
Then the digest for RexCall could be the bits relevant to RexCall itself +
digests of the operands (which can be reused as is)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3767) AssertionError when SqlBinaryStringLiteral appears in tablesample substitute
Vladimir Sitnikov created CALCITE-3767:
--
Summary: AssertionError when SqlBinaryStringLiteral appears in
tablesample substitute
Key: CALCITE-3767
URL: https://issues.apache.org/jira/browse/CALCITE-3767
Project: Calcite
Issue Type: Bug
Reporter: Vladimir Sitnikov
SQL:
{code:sql}select * from dual tablesample substitute(x'');{code}
Note: I have absolutely no clue what that SQL means :)
However, it might have better error reporting.
{noformat}
!connect jdbc:calcite:fun=spatial,oracle "sa" ""
0: jdbc:calcite:fun=spatial,oracle> select * from dual tablesample
substitute(x'');
Error: Error while executing SQL "select * from dual tablesample
substitute(x'')": parse failed: cannot cast as class java.lang.String
(state=,code=0)
{noformat}
{noformat}
java.lang.AssertionError: cannot cast as class java.lang.String
at org.apache.calcite.sql.SqlLiteral.getValueAs(SqlLiteral.java:351)
at
org.javacc.test.calcite.SqlParserImplFuzzyParser.TableRef2(SqlParserImplFuzzyParser.java:2698)
at
org.javacc.test.calcite.SqlParserImplFuzzyParser.TableRef(SqlParserImplFuzzyParser.java:2557)
at
org.javacc.test.calcite.SqlParserImplFuzzyParser.FromClause(SqlParserImplFuzzyParser.java:2442)
at
org.javacc.test.calcite.SqlParserImplFuzzyParser.SqlSelect(SqlParserImplFuzzyParser.java:1552)
...
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3750) PigRelBuilderStyleTest fails with ConcurrentModificationException
Vladimir Sitnikov created CALCITE-3750:
--
Summary: PigRelBuilderStyleTest fails with
ConcurrentModificationException
Key: CALCITE-3750
URL: https://issues.apache.org/jira/browse/CALCITE-3750
Project: Calcite
Issue Type: Bug
Components: pig-adapter
Affects Versions: 1.21.0
Reporter: Vladimir Sitnikov
{noformat}
PigRelBuilderStyleTest > testImplWithCountWithoutGroupBy() FAILED
java.util.ConcurrentModificationException
at java.util.Hashtable$Enumerator.next(Hashtable.java:1387)
at
org.apache.pig.impl.util.PropertiesUtil.loadDefaultProperties(PropertiesUtil.java:65)
at
org.apache.pig.impl.util.PropertiesUtil.loadDefaultProperties(PropertiesUtil.java:167)
at org.apache.pig.ExecTypeProvider.fromString(ExecTypeProvider.java:76)
at org.apache.pig.test.Util.getLocalTestMode(Util.java:1395)
at
org.apache.calcite.test.PigRelBuilderStyleTest.setupDataFilesForPigServer(PigRelBuilderStyleTest.java:286)
FAILURE 0.5sec, org.apache.calcite.test.PigRelBuilderStyleTest >
testScanAndFilter()
{noformat}
{noformat}
PigRelBuilderStyleTest > testScanAndFilter() FAILED
java.util.ConcurrentModificationException
at java.util.Hashtable$Enumerator.next(Hashtable.java:1387)
at
org.apache.pig.impl.util.PropertiesUtil.loadDefaultProperties(PropertiesUtil.java:65)
at
org.apache.pig.impl.util.PropertiesUtil.loadDefaultProperties(PropertiesUtil.java:167)
at org.apache.pig.ExecTypeProvider.fromString(ExecTypeProvider.java:76)
at org.apache.pig.test.Util.getLocalTestMode(Util.java:1395)
at
org.apache.calcite.test.PigRelBuilderStyleTest.setupDataFilesForPigServer(PigRelBuilderStyleTest.java:286)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3742) Update Gradle: 6.0.1 -> 6.1
Vladimir Sitnikov created CALCITE-3742: -- Summary: Update Gradle: 6.0.1 -> 6.1 Key: CALCITE-3742 URL: https://issues.apache.org/jira/browse/CALCITE-3742 Project: Calcite Issue Type: Improvement Affects Versions: 1.21.0 Reporter: Vladimir Sitnikov Assignee: Vladimir Sitnikov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3725) RelMetadataTest fails with NPE due to unsafe RelMetadataQuery.instance call
Vladimir Sitnikov created CALCITE-3725:
--
Summary: RelMetadataTest fails with NPE due to unsafe
RelMetadataQuery.instance call
Key: CALCITE-3725
URL: https://issues.apache.org/jira/browse/CALCITE-3725
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.21.0
Reporter: Vladimir Sitnikov
{noformat}
org.apache.calcite.test.RelMetadataTest >
testColumnUniquenessForMinusWithConstantColumns() FAILED
java.lang.NullPointerException
at java.util.Objects.requireNonNull(Objects.java:203)
at
org.apache.calcite.rel.metadata.RelMetadataQuery.(RelMetadataQuery.java:141)
at
org.apache.calcite.rel.metadata.RelMetadataQuery.(RelMetadataQuery.java:106)
at
org.apache.calcite.rel.metadata.RelMetadataQuery.instance(RelMetadataQuery.java:174)
at
org.apache.calcite.test.RelMetadataTest.testColumnUniquenessForMinusWithConstantColumns(RelMetadataTest.java:1057)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3722) Add Hook#PLAN_BEFORE_IMPLEMENTATION to capture the plan after optimization
Vladimir Sitnikov created CALCITE-3722: -- Summary: Add Hook#PLAN_BEFORE_IMPLEMENTATION to capture the plan after optimization Key: CALCITE-3722 URL: https://issues.apache.org/jira/browse/CALCITE-3722 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.21.0 Reporter: Vladimir Sitnikov Assignee: Vladimir Sitnikov This helps to validate the execution plan, especially when sql-to-rel is implemented with a custom factory as in CalciteAssert#withRel. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3713) Remove column names from Project#digest
Vladimir Sitnikov created CALCITE-3713:
--
Summary: Remove column names from Project#digest
Key: CALCITE-3713
URL: https://issues.apache.org/jira/browse/CALCITE-3713
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.21.0
Reporter: Vladimir Sitnikov
Assignee: Vladimir Sitnikov
Projects that differ with column names only are equivalent, so the planning
space can be reduced if field names do not participate in relation matching.
For instance, {{Project(input=rel#1,empid=$0)}} and
{{Project(input=rel#1,deptno=$0)}} are equivalent
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3712) Optimize lossless casts in RexSimplify: CAST(CAST(intExpr as BIGINT) as INT) => intExpr
Vladimir Sitnikov created CALCITE-3712: -- Summary: Optimize lossless casts in RexSimplify: CAST(CAST(intExpr as BIGINT) as INT) => intExpr Key: CALCITE-3712 URL: https://issues.apache.org/jira/browse/CALCITE-3712 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.21.0 Reporter: Vladimir Sitnikov Assignee: Vladimir Sitnikov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3710) MaterializationService#defineMaterialization should inherit connection properties
Vladimir Sitnikov created CALCITE-3710:
--
Summary: MaterializationService#defineMaterialization should
inherit connection properties
Key: CALCITE-3710
URL: https://issues.apache.org/jira/browse/CALCITE-3710
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.21.0
Reporter: Vladimir Sitnikov
Currently, defineMaterialization uses
{code:java}
final CalciteConnection connection =
CalciteMetaImpl.connect(schema.root(), null);
{code}
so it does not inherit the connection properties (quoting, caseSensitive, etc).
Steps to reproduce: add {{.with(Lex.JAVA)}} to MaterializationTest (e.g.
testPrePopulated)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3709) Use "rejected row count" for RelOptCost#getRows
Vladimir Sitnikov created CALCITE-3709: -- Summary: Use "rejected row count" for RelOptCost#getRows Key: CALCITE-3709 URL: https://issues.apache.org/jira/browse/CALCITE-3709 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Vladimir Sitnikov Current cost#rows has a problem: it does not add well when computing cumulative cost. So the idea is to use the number of _rejected_ rows. Then the field would have certain meaning: * If the value is high, the plan is probably rejecting a lot of unrelated rows, thus it is suboptimal * Extra Project/Calc nodes won't artificially increase rows in the cost fields. Currently each Project adds "rows" which is not very good. * It is clear what to put to the rows field: "rejected rows" is more-or-less understandable. For Project it would be 0. * Join/Filter/Calc nodes would show "estimated number of returned rows=X (from metadataquery), rejected rows=Y (from cost)" which would help understanding where the time is spent That is inspired by PostgreSQL's "rows removed by filter" when running explain analyze (which is statement execution + collecting statistics on each execution plan node): http://wiki.postgresql.org/wiki/What's_new_in_PostgreSQL_9.2#Explain_improvements -- This message was sent by Atlassian Jira (v8.3.4#803005)
