Re: [jira] [Created] (CALCITE-4898) Upgrading Elasticsearch version from 7.0.1 to 7.15 in Elasticsearch Adapter

[Julian Hyde]

This is a useful change (just committed in 
https://github.com/apache/calcite/commit/d6a36fc08715eba31d1d6bc26ef1ff5558d04b8d
 
)
 but it introduces a new package repository.

Are we OK with that?

The ConJars.org repository (which I introduced, years ago) was a pain to get 
rid of. The repository went offline, and though we had stopped using it several 
releases ago, it was still referenced in old Calcite releases. The net effect 
was that Hive was unable to build.

The lesson I learned was that a package repository is a long-term commitment. 
If it’s not rock-solid and there for the long-term, we and dependent projects 
are going to get suffer flaky and broken builds that we are powerless to fix. 

In my opinion, only two repositories can be bet upon long-term: Apache’s 
repository and Maven Central.

It looks as if the necessary module is in Maven Central: 
https://search.maven.org/search?q=scripting-painless-spi 
.

Julian


> On Nov 23, 2021, at 6:36 AM, ZheHu (Jira)  wrote:
> 
> ZheHu created CALCITE-4898:
> --
> 
> Summary: Upgrading Elasticsearch version from 7.0.1 to 7.15 in 
> Elasticsearch Adapter
> Key: CALCITE-4898
> URL: https://issues.apache.org/jira/browse/CALCITE-4898
> Project: Calcite
>  Issue Type: Improvement
>  Components: elasticsearch-adapter
>Affects Versions: 1.28.0
>Reporter: ZheHu
>Assignee: ZheHu
> 
> 
> The current Elasticsearch version in Calcite is 7.0.1, while the latest 
> released ES version is 7.15.2.
> 
> There are many new SQL features added and bugs fixed from 7.0.1 to 7.15. By 
> upgrading the embedded ES, we can make good use of ES's new features, 
> especially about DSL, which enable developers to find one more suitable or 
> effective way to implement features and improve existing functions.
> 
> Since series of 7.x versions have good compatibility, this issue will just 
> upgrade ES server without adjusting RestClient version in Elasticsearch 
> Adapter.
> 
> 
> 
> 
> 
> --
> This message was sent by Atlassian Jira
> (v8.20.1#820001)

[jira] [Created] (CALCITE-4945) Compilation failes on subquery

[Vladimir Steshin (Jira)]

Vladimir Steshin created CALCITE-4945:
-

 Summary: Compilation failes on subquery
 Key: CALCITE-4945
 URL: https://issues.apache.org/jira/browse/CALCITE-4945
 Project: Calcite
  Issue Type: Bug
Reporter: Vladimir Steshin


While researching CALCITE-685 (calcite 1.29, master branch), I met an 
calcite-runtime compilation error.

Request example:
{code:sql}
select e.department_id, (select 1 from employee e2 where e.department_id = 
-100) from employee e;
{code}

Stacktrace:
{code:java}
2021-12-16 21:42:38,032 [ForkJoinPool-1-worker-9] INFO  - open start - state 
modified
2021-12-16 21:42:43,612 [ForkJoinPool-1-worker-9] INFO  - Checkpoint start
2021-12-16 21:42:43,613 [ForkJoinPool-1-worker-9] INFO  - Checkpoint end - 
txts: 279

Error while executing SQL "select e.department_id,
   ( select e2.employee_id
 from  employee e2
 where e.department_id = 1000
   )
from employee e
": Error while compiling generated Java code:
public static class Record2_0 implements java.io.Serializable {
  public boolean f0;
  public int f1;
  public Record2_0() {}
  public boolean equals(Object o) {
if (this == o) {
  return true;
}
if (!(o instanceof Record2_0)) {
  return false;
}
return this.f0 == ((Record2_0) o).f0 && this.f1 == ((Record2_0) o).f1;
  }

  public int hashCode() {
int h = 0;
h = org.apache.calcite.runtime.Utilities.hash(h, this.f0);
h = org.apache.calcite.runtime.Utilities.hash(h, this.f1);
return h;
  }

  public int compareTo(Record2_0 that) {
int c;
c = org.apache.calcite.runtime.Utilities.compare(this.f0, that.f0);
if (c != 0) {
  return c;
}
c = org.apache.calcite.runtime.Utilities.compare(this.f1, that.f1);
if (c != 0) {
  return c;
}
return 0;
  }

  public String toString() {
return "{f0=" + this.f0 + ", f1=" + this.f1 + "}";
  }

}

public org.apache.calcite.linq4j.Enumerable bind(final 
org.apache.calcite.DataContext root) {
  final org.apache.calcite.linq4j.Enumerable _inputEnumerable = 
org.apache.calcite.schema.Schemas.queryable(root, 
root.getRootSchema().getSubSchema("foodmart2"), java.lang.Object[].class, 
"employee").asEnumerable();
  final org.apache.calcite.linq4j.AbstractEnumerable child = new 
org.apache.calcite.linq4j.AbstractEnumerable(){
public org.apache.calcite.linq4j.Enumerator enumerator() {
  return new org.apache.calcite.linq4j.Enumerator(){
  public final org.apache.calcite.linq4j.Enumerator inputEnumerator = 
_inputEnumerable.enumerator();
  public void reset() {
inputEnumerator.reset();
  }

  public boolean moveNext() {
return inputEnumerator.moveNext();
  }

  public void close() {
inputEnumerator.close();
  }

  public Object current() {
final Object[] current = (Object[]) inputEnumerator.current();
return new Object[] {
current[0],
current[7],
org.apache.calcite.runtime.SqlFunctions.toInt(current[7]) == 
1000};
  }

};
}

  };
  final org.apache.calcite.linq4j.Enumerable _inputEnumerable0 = 
org.apache.calcite.schema.Schemas.queryable(root, 
root.getRootSchema().getSubSchema("foodmart2"), java.lang.Object[].class, 
"employee").asEnumerable();
  final org.apache.calcite.linq4j.AbstractEnumerable left0 = new 
org.apache.calcite.linq4j.AbstractEnumerable(){
public org.apache.calcite.linq4j.Enumerator enumerator() {
  return new org.apache.calcite.linq4j.Enumerator(){
  public final org.apache.calcite.linq4j.Enumerator inputEnumerator = 
_inputEnumerable0.enumerator();
  public void reset() {
inputEnumerator.reset();
  }

  public boolean moveNext() {
return inputEnumerator.moveNext();
  }

  public void close() {
inputEnumerator.close();
  }

  public Object current() {
return org.apache.calcite.runtime.SqlFunctions.toInt(((Object[]) 
inputEnumerator.current())[0]);
  }

};
}

  };
  final org.apache.calcite.linq4j.Enumerable _inputEnumerable00 = 
org.apache.calcite.schema.Schemas.queryable(root, 
root.getRootSchema().getSubSchema("foodmart2"), java.lang.Object[].class, 
"employee").asEnumerable();
  final org.apache.calcite.linq4j.AbstractEnumerable child0 = new 
org.apache.calcite.linq4j.AbstractEnumerable(){
public org.apache.calcite.linq4j.Enumerator enumerator() {
  return new org.apache.calcite.linq4j.Enumerator(){
  public final org.apache.calcite.linq4j.Enumerator inputEnumerator = 
_inputEnumerable00.enumerator();
  public void reset() {
inputEnumerator.reset();
  }

  public boolean moveNext() {
while (inputEnumerator.moveNext()) {
  

Creating a parser for Big query

[Arash Bizhan zadeh]

Hi,
I am trying to create a parser to parse big query statements like this -
this is kotlin btw:

val sqlParserConfig = BigQuerySqlDialect(
BigQuerySqlDialect.DEFAULT_CONTEXT.withDatabaseProduct(SqlDialect.DatabaseProduct.BIG_QUERY)).configureParser(SqlParser.config()
  .withParserFactory(SqlBabelParserImpl.FACTORY)

  .withConformance(SqlConformanceEnum.BIG_QUERY))
val parser = SqlParser.create(str, sqlParserConfig)

but the resulting parser seems not to understand BQ specific things like
the `IF` function.

How can I properly construct a SqlParser?

Jenkins build is back to normal : Calcite » Calcite-snapshots #16

[Apache Jenkins Server]

See 

Re: Failing task :linq4j:jandexMain when executing CheckerFramework

[Vladimir Sitnikov]

>First some annoying :core:checkstyleTest that interferes with files
generated by Intellij IDEA, with hundreds of error messages similar to the
following:

We exclude generated files from the style checks via the following patterns:
https://github.com/apache/calcite/blob/a6293a37dd4166e6fe69ae0c5ef7a89e6677b6ab/build.gradle.kts#L259-L276

Please add the relevant pattern and file the PR.
However, it is really strange that you get the generated files into
generated/sources/version/generated and javacc/javaCCTest/generated_tests
folders.
I don't understand why IDEA would generate files into those folders.
It does not sound like your ImmutableMyProjectFilterRuleConfig is connected
with javaCCTest.
I would try to understand why do you get files into /javaCCTest/ and
/version/ folders.

A side note: Kotlin does have the official coding style, so there's a
formatter that yields nice results and that is consistent with IDEs.
So moving to Kotlin makes many Checkstyle rules obsolete. No joking.

>Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds
for length 0

It is very likely you are using an outdated version of javac which produced
invalid bytecode.
Please upgrade javac (e.g. to the recent patchset).
Note: GitHub CI uses recent patchet updates, so you might fail to reproduce
the issue.

We do know javac versions 1.8 before u202 are extremely buggy, so we fail
the build for those versions.
It might be time to add the same verification for 11, and require 11.0.5 or
later (in case of 11).

>Is there an easy way to fix the problem?

The easy fix is to bump the javac version.
The funny coincidence is that Kotlin solves this issue as well: null safety
is integrated into the regular compilation,
so there's no need for a separate (and time-consuming!) checker framework
compilation.

Vladimir

[jira] [Created] (CALCITE-4944) Add a property to activate the RuleMatchVisualizer

[Thomas Rebele (Jira)]

Thomas Rebele created CALCITE-4944:
--

 Summary: Add a property to activate the RuleMatchVisualizer
 Key: CALCITE-4944
 URL: https://issues.apache.org/jira/browse/CALCITE-4944
 Project: Calcite
  Issue Type: New Feature
Reporter: Thomas Rebele


As discussed in 
[CALCITE-4737|https://issues.apache.org/jira/browse/CALCITE-4737?focusedCommentId=17459849#comment-17459849],
 a property to activate the RuleMatchVisualizer should be implemented. To be 
investigated whether the common parts of the generated html files should me 
moved to different files (css, js), and reused for all generated files.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (CALCITE-4943) Type conversin in Project / HepPlaner: INTEGER to JavaType(int)

[Vladimir Steshin (Jira)]

Vladimir Steshin created CALCITE-4943:
-

 Summary: Type conversin in Project / HepPlaner: INTEGER to 
JavaType(int)
 Key: CALCITE-4943
 URL: https://issues.apache.org/jira/browse/CALCITE-4943
 Project: Calcite
  Issue Type: Bug
Reporter: Vladimir Steshin


Found by JdbcTest.

Test query:
{code:sql}
select (select (select t."empid")) from "hr"."emps" t
{code}

Error:
{code:java}
java.lang.AssertionError: type mismatch:
ref:
JavaType(int) NOT NULL
input:
INTEGER NOT NULL
{code}

Stacktrace:

{code:java}
at org.apache.calcite.util.Litmus$1.fail(Litmus.java:32)
at org.apache.calcite.plan.RelOptUtil.eq(RelOptUtil.java:2211)
at org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:129)
at org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:61)
at org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:114)
at org.apache.calcite.rex.RexChecker.visitCall(RexChecker.java:144)
at org.apache.calcite.rex.RexChecker.visitCall(RexChecker.java:61)
at org.apache.calcite.rex.RexCall.accept(RexCall.java:189)
at org.apache.calcite.rel.core.Project.isValid(Project.java:219)
at org.apache.calcite.rel.core.Project.(Project.java:98)
at 
org.apache.calcite.rel.logical.LogicalProject.(LogicalProject.java:69)
at 
org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:126)
at 
org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:114)
at 
org.apache.calcite.rel.core.RelFactories$ProjectFactoryImpl.createProject(RelFactories.java:178)
at org.apache.calcite.tools.RelBuilder.project_(RelBuilder.java:2020)
at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1792)
at 
org.apache.calcite.tools.RelBuilder.projectNamed(RelBuilder.java:2083)
at 
org.apache.calcite.sql2rel.RelDecorrelator.projectJoinOutputWithNullability(RelDecorrelator.java:1445)
at 
org.apache.calcite.sql2rel.RelDecorrelator.access$900(RelDecorrelator.java:147)
at 
org.apache.calcite.sql2rel.RelDecorrelator$RemoveCorrelationForScalarProjectRule.onMatch(RelDecorrelator.java:2149)
at 
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:343)
at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:565)
at 
org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:428)
at 
org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:251)
at 
org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:130)
at 
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:208)
at 
org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:195)
at 
org.apache.calcite.sql2rel.RelDecorrelator.removeCorrelationViaRule(RelDecorrelator.java:378)
at 
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:221)
at 
org.apache.calcite.tools.Programs$DecorrelateProgram.run(Programs.java:361)
at 
org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:336)
at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:177)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:312)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)
at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:647)
at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:513)
at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:483)
{code}





--
This message was sent by Atlassian Jira
(v8.20.1#820001)