[jira] [Created] (CALCITE-2246) column alias in view will cause duplicate column exception

[Yongqian Wang (JIRA)]

Yongqian Wang created CALCITE-2246:
--

 Summary: column alias in view will cause duplicate column 
exception 
 Key: CALCITE-2246
 URL: https://issues.apache.org/jira/browse/CALCITE-2246
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.16.0
 Environment: using mysql 5.7 database
Reporter: Yongqian Wang
Assignee: Julian Hyde


 

I have a view like this , testdb is a mysql datasource , i have a view with a 
join statements
{code:java}
  "name": "test",
  "type": "view",
  "sql": [
"select t0.\"id\" as \"id\", t0.\"name\" as name from TESTDB.\"a\" t0 
left join TESTDB.\"b\" t1 on t0.\"id\" = t1.\"id\""  
{code}
when i try to execute query like "select * from test.view limit 1", i got this 
exception
{code:java}
java.sql.SQLException: exception while executing query: while executing SQL 
[SELECT `t1`.`id`, `t1`.`name` AS `NAME`
FROM (SELECT *
FROM (SELECT *
FROM `a`) AS `t`
LEFT JOIN (SELECT `id`
FROM `b`) AS `t0` ON `t`.`id` = `t0`.`id`) AS `t1`]

Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Duplicate 
column name 'id'
{code}
 

which t1 return two "id" column



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: JDK 8 syntax

[Enrico Olivelli]

This is "the" error (see below). Unfortunately just be upgrading Janino to
3.0.8 the problem still is present.

the only 'unwrap'  word in the generated code is here
...
public org.apache.calcite.linq4j.Enumerable bind(final
org.apache.calcite.DataContext root0) {
  root = root0;
  final org.apache.calcite.linq4j.Enumerable _inputEnumerable =
org.apache.calcite.linq4j.Linq4j.asEnumerable(((org.apache.calcite.test.ReflectiveSchemaTest.CatchallSchema)
((org.apache.calcite.adapter.java.ReflectiveSchema)
root.getRootSchema().getSubSchema("s").unwrap(org.apache.calcite.adapter.java.ReflectiveSchema.class)).getTarget()).nullables);


Does any ring bell ?

Enrico


Caused by: org.codehaus.commons.compiler.CompileException: Line 5, Column
272: A method named "unwrap" is not declared in any enclosing class nor any
supertype, nor through a static import
> at
org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:11821)
> at
org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:8731)
> at
org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4737)
> at org.codehaus.janino.UnitCompiler.access$8300(UnitCompiler.java:212)
> at
org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(UnitCompiler.java:4097)
> at
org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(UnitCompiler.java:4070)
> at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:4902)
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4070)
> at
org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5253)
> at
org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4696)
> at org.codehaus.janino.UnitCompiler.access$7800(UnitCompiler.java:212)
> at
org.codehaus.janino.UnitCompiler$12.visitCast(UnitCompiler.java:4092)
> at
org.codehaus.janino.UnitCompiler$12.visitCast(UnitCompiler.java:4070)
> at org.codehaus.janino.Java$Cast.accept(Java.java:4727)
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4070)
> at
org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4732)
> at org.codehaus.janino.UnitCompiler.access$7300(UnitCompiler.java:212)
> at
org.codehaus.janino.UnitCompiler$12$1.visitParenthesizedExpression(UnitCompiler.java:4083)
> at
org.codehaus.janino.UnitCompiler$12$1.visitParenthesizedExpression(UnitCompiler.java:4074)
> at
org.codehaus.janino.Java$ParenthesizedExpression.accept(Java.java:4753)
> at
org.codehaus.janino.UnitCompiler$12.visitLvalue(UnitCompiler.java:4074)
> at
org.codehaus.janino.UnitCompiler$12.visitLvalue(UnitCompiler.java:4070)
> at org.codehaus.janino.Java$Lvalue.accept(Java.java:3977)
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4070)
> at
org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5253)
> at
org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4792)
> at org.codehaus.janino.UnitCompiler.access$8300(UnitCompiler.java:212)
> at
org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(UnitCompiler.java:4097)
> at
org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(UnitCompiler.java:4070)
> at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:4902)
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4070)
> at
org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5253)
> at
org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4696)
> at org.codehaus.janino.UnitCompiler.access$7800(UnitCompiler.java:212)
> at
org.codehaus.janino.UnitCompiler$12.visitCast(UnitCompiler.java:4092)
> at
org.codehaus.janino.UnitCompiler$12.visitCast(UnitCompiler.java:4070)
> at org.codehaus.janino.Java$Cast.accept(Java.java:4727)
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4070)
> at
org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4732)
> at org.codehaus.janino.UnitCompiler.access$7300(UnitCompiler.java:212)
> at
org.codehaus.janino.UnitCompiler$12$1.visitParenthesizedExpression(UnitCompiler.java:4083)
> at
org.codehaus.janino.UnitCompiler$12$1.visitParenthesizedExpression(UnitCompiler.java:4074)
> at
org.codehaus.janino.Java$ParenthesizedExpression.accept(Java.java:4753)
> at
org.codehaus.janino.UnitCompiler$12.visitLvalue(UnitCompiler.java:4074)
> at
org.codehaus.janino.UnitCompiler$12.visitLvalue(UnitCompiler.java:4070)
> at org.codehaus.janino.Java$Lvalue.accept(Java.java:3977)
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4070)
> at
org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5253)
> at
org.codehaus.janino.UnitCompiler.compileContext2(UnitCompiler.java:4010)
> at org.codehaus.janino.UnitCompiler.access$6100(UnitCompiler.java:212)
> at
org.codehaus.janino.UnitCompiler$11$1.visitFieldAccess(UnitCompiler.java:3947)
> at

Re: JDK 8 syntax

[Enrico Olivelli]

2018-04-10 13:48 GMT+02:00 Julian Hyde :

> Yes, that’s it.
>
> Next step would be to check whether the generated code is the same as
> previously, and if so, it is a Janino bug. Then try to create a minimal
> test case for Janino. Maybe we can work around by adding an explicit cast
> somewhere.
>
> Thanks for reproducing this, and trying the Janino upgrade.
>


Julian,
If you have no hurry for this I will be happy to spend some time in order
to figure out the root cause.

I will follow the discussion and jump in when I have some idea.

It will be great to have Calcite really built on jdk8 and leverage lambdas


Cheers
Enrico



>
> Julian
>
> > On Apr 10, 2018, at 3:37 AM, Enrico Olivelli 
> wrote:
> >
> > This is "the" error (see below). Unfortunately just be upgrading Janino
> to
> > 3.0.8 the problem still is present.
> >
> > the only 'unwrap'  word in the generated code is here
> > ...
> > public org.apache.calcite.linq4j.Enumerable bind(final
> > org.apache.calcite.DataContext root0) {
> >  root = root0;
> >  final org.apache.calcite.linq4j.Enumerable _inputEnumerable =
> > org.apache.calcite.linq4j.Linq4j.asEnumerable(((org.apache.calcite.test.
> ReflectiveSchemaTest.CatchallSchema)
> > ((org.apache.calcite.adapter.java.ReflectiveSchema)
> > root.getRootSchema().getSubSchema("s").unwrap(org.
> apache.calcite.adapter.java.ReflectiveSchema.class)).
> getTarget()).nullables);
> > 
> >
> > Does any ring bell ?
> >
> > Enrico
> >
> >
> > Caused by: org.codehaus.commons.compiler.CompileException: Line 5,
> Column
> > 272: A method named "unwrap" is not declared in any enclosing class nor
> any
> > supertype, nor through a static import
> >>at
> > org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:11821)
> >>at
> > org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:8731)
> >>at
> > org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4737)
> >>at org.codehaus.janino.UnitCompiler.access$8300(
> UnitCompiler.java:212)
> >>at
> > org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(
> UnitCompiler.java:4097)
> >>at
> > org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(
> UnitCompiler.java:4070)
> >>at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:4902)
> >>at org.codehaus.janino.UnitCompiler.compileGet(
> UnitCompiler.java:4070)
> >>at
> > org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5253)
> >>at
> > org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4696)
> >>at org.codehaus.janino.UnitCompiler.access$7800(
> UnitCompiler.java:212)
> >>at
> > org.codehaus.janino.UnitCompiler$12.visitCast(UnitCompiler.java:4092)
> >>at
> > org.codehaus.janino.UnitCompiler$12.visitCast(UnitCompiler.java:4070)
> >>at org.codehaus.janino.Java$Cast.accept(Java.java:4727)
> >>at org.codehaus.janino.UnitCompiler.compileGet(
> UnitCompiler.java:4070)
> >>at
> > org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4732)
> >>at org.codehaus.janino.UnitCompiler.access$7300(
> UnitCompiler.java:212)
> >>at
> > org.codehaus.janino.UnitCompiler$12$1.visitParenthesizedExpression(
> UnitCompiler.java:4083)
> >>at
> > org.codehaus.janino.UnitCompiler$12$1.visitParenthesizedExpression(
> UnitCompiler.java:4074)
> >>at
> > org.codehaus.janino.Java$ParenthesizedExpression.accept(Java.java:4753)
> >>at
> > org.codehaus.janino.UnitCompiler$12.visitLvalue(UnitCompiler.java:4074)
> >>at
> > org.codehaus.janino.UnitCompiler$12.visitLvalue(UnitCompiler.java:4070)
> >>at org.codehaus.janino.Java$Lvalue.accept(Java.java:3977)
> >>at org.codehaus.janino.UnitCompiler.compileGet(
> UnitCompiler.java:4070)
> >>at
> > org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5253)
> >>at
> > org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4792)
> >>at org.codehaus.janino.UnitCompiler.access$8300(
> UnitCompiler.java:212)
> >>at
> > org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(
> UnitCompiler.java:4097)
> >>at
> > org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(
> UnitCompiler.java:4070)
> >>at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:4902)
> >>at org.codehaus.janino.UnitCompiler.compileGet(
> UnitCompiler.java:4070)
> >>at
> > org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5253)
> >>at
> > org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4696)
> >>at org.codehaus.janino.UnitCompiler.access$7800(
> UnitCompiler.java:212)
> >>at
> > org.codehaus.janino.UnitCompiler$12.visitCast(UnitCompiler.java:4092)
> >>at
> > org.codehaus.janino.UnitCompiler$12.visitCast(UnitCompiler.java:4070)
> >>at org.codehaus.janino.Java$Cast.accept(Java.java:4727)
> >>at org.codehaus.janino.UnitCompiler.compileGet(
> UnitCompiler.java:4070)
> >>at
> > 

Re: JDK 8 syntax

[Julian Hyde]

I’d really appreciate that.

Can you please log a JIRA case, and record your observations/progress there.

There’s no particular hurry on this one. 

Julian

> On Apr 10, 2018, at 06:32, Enrico Olivelli  wrote:
> 
> 2018-04-10 13:48 GMT+02:00 Julian Hyde :
> 
>> Yes, that’s it.
>> 
>> Next step would be to check whether the generated code is the same as
>> previously, and if so, it is a Janino bug. Then try to create a minimal
>> test case for Janino. Maybe we can work around by adding an explicit cast
>> somewhere.
>> 
>> Thanks for reproducing this, and trying the Janino upgrade.
>> 
> 
> 
> Julian,
> If you have no hurry for this I will be happy to spend some time in order
> to figure out the root cause.
> 
> I will follow the discussion and jump in when I have some idea.
> 
> It will be great to have Calcite really built on jdk8 and leverage lambdas
> 
> 
> Cheers
> Enrico
> 
> 
> 
>> 
>> Julian
>> 
>>> On Apr 10, 2018, at 3:37 AM, Enrico Olivelli 
>> wrote:
>>> 
>>> This is "the" error (see below). Unfortunately just be upgrading Janino
>> to
>>> 3.0.8 the problem still is present.
>>> 
>>> the only 'unwrap'  word in the generated code is here
>>> ...
>>> public org.apache.calcite.linq4j.Enumerable bind(final
>>> org.apache.calcite.DataContext root0) {
>>> root = root0;
>>> final org.apache.calcite.linq4j.Enumerable _inputEnumerable =
>>> org.apache.calcite.linq4j.Linq4j.asEnumerable(((org.apache.calcite.test.
>> ReflectiveSchemaTest.CatchallSchema)
>>> ((org.apache.calcite.adapter.java.ReflectiveSchema)
>>> root.getRootSchema().getSubSchema("s").unwrap(org.
>> apache.calcite.adapter.java.ReflectiveSchema.class)).
>> getTarget()).nullables);
>>> 
>>> 
>>> Does any ring bell ?
>>> 
>>> Enrico
>>> 
>>> 
>>> Caused by: org.codehaus.commons.compiler.CompileException: Line 5,
>> Column
>>> 272: A method named "unwrap" is not declared in any enclosing class nor
>> any
>>> supertype, nor through a static import
   at
>>> org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:11821)
   at
>>> org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:8731)
   at
>>> org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4737)
   at org.codehaus.janino.UnitCompiler.access$8300(
>> UnitCompiler.java:212)
   at
>>> org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(
>> UnitCompiler.java:4097)
   at
>>> org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(
>> UnitCompiler.java:4070)
   at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:4902)
   at org.codehaus.janino.UnitCompiler.compileGet(
>> UnitCompiler.java:4070)
   at
>>> org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5253)
   at
>>> org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4696)
   at org.codehaus.janino.UnitCompiler.access$7800(
>> UnitCompiler.java:212)
   at
>>> org.codehaus.janino.UnitCompiler$12.visitCast(UnitCompiler.java:4092)
   at
>>> org.codehaus.janino.UnitCompiler$12.visitCast(UnitCompiler.java:4070)
   at org.codehaus.janino.Java$Cast.accept(Java.java:4727)
   at org.codehaus.janino.UnitCompiler.compileGet(
>> UnitCompiler.java:4070)
   at
>>> org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4732)
   at org.codehaus.janino.UnitCompiler.access$7300(
>> UnitCompiler.java:212)
   at
>>> org.codehaus.janino.UnitCompiler$12$1.visitParenthesizedExpression(
>> UnitCompiler.java:4083)
   at
>>> org.codehaus.janino.UnitCompiler$12$1.visitParenthesizedExpression(
>> UnitCompiler.java:4074)
   at
>>> org.codehaus.janino.Java$ParenthesizedExpression.accept(Java.java:4753)
   at
>>> org.codehaus.janino.UnitCompiler$12.visitLvalue(UnitCompiler.java:4074)
   at
>>> org.codehaus.janino.UnitCompiler$12.visitLvalue(UnitCompiler.java:4070)
   at org.codehaus.janino.Java$Lvalue.accept(Java.java:3977)
   at org.codehaus.janino.UnitCompiler.compileGet(
>> UnitCompiler.java:4070)
   at
>>> org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5253)
   at
>>> org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4792)
   at org.codehaus.janino.UnitCompiler.access$8300(
>> UnitCompiler.java:212)
   at
>>> org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(
>> UnitCompiler.java:4097)
   at
>>> org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(
>> UnitCompiler.java:4070)
   at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:4902)
   at org.codehaus.janino.UnitCompiler.compileGet(
>> UnitCompiler.java:4070)
   at
>>> org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5253)
   at
>>> org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4696)
   at org.codehaus.janino.UnitCompiler.access$7800(
>> UnitCompiler.java:212)
   at
>>> 

[jira] [Created] (CALCITE-2248) Take advantage of Calcite DDL

[Andrew Pilloud (JIRA)]

Andrew Pilloud created CALCITE-2248:
---

 Summary: Take advantage of Calcite DDL
 Key: CALCITE-2248
 URL: https://issues.apache.org/jira/browse/CALCITE-2248
 Project: Calcite
  Issue Type: New Feature
Reporter: Andrew Pilloud
Assignee: Julian Hyde


In Calcite 1.15 abstract support for DDL moved into the calcite core. We should 
take advantage of that.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Calcite-Master - Build # 249 - Failure

[Apache Jenkins Server]

The Apache Jenkins build system has built Calcite-Master (build #249)

Status: Failure

Check console output at https://builds.apache.org/job/Calcite-Master/249/ to 
view the results.

Calcite-Avatica-Master - Build # 49 - Failure

[Apache Jenkins Server]

The Apache Jenkins build system has built Calcite-Avatica-Master (build #49)

Status: Failure

Check console output at 
https://builds.apache.org/job/Calcite-Avatica-Master/49/ to view the results.

Calcite-Master - Build # 250 - Still Failing

[Apache Jenkins Server]

The Apache Jenkins build system has built Calcite-Master (build #250)

Status: Still Failing

Check console output at https://builds.apache.org/job/Calcite-Master/250/ to 
view the results.

Re: Use of CalciteSchema across Calcite

[Laurent Goujon]

That was my understanding, but it seems usage of CalciteSchema is prevalent
outside the scope of the JDBC adapter:

$ grep -rl import.*CalciteSchema core/src/main/java|grep -v calcite/jdbc
core/src/main/java/org/apache/calcite/materialize/Lattice.java
core/src/main/java/org/apache/calcite/materialize/MaterializationActor.java
core/src/main/java/org/apache/calcite/materialize/MaterializationService.java
core/src/main/java/org/apache/calcite/model/ModelHandler.java
core/src/main/java/org/apache/calcite/plan/RelOptLattice.java
core/src/main/java/org/apache/calcite/prepare/CalciteCatalogReader.java
core/src/main/java/org/apache/calcite/prepare/CalciteMaterializer.java
core/src/main/java/org/apache/calcite/prepare/CalcitePrepareImpl.java
core/src/main/java/org/apache/calcite/prepare/PlannerImpl.java
core/src/main/java/org/apache/calcite/prepare/Prepare.java
core/src/main/java/org/apache/calcite/prepare/QueryableRelBuilder.java
core/src/main/java/org/apache/calcite/prepare/RelOptTableImpl.java
core/src/main/java/org/apache/calcite/rel/rules/AggregateStarTableRule.java
core/src/main/java/org/apache/calcite/schema/impl/MaterializedViewTable.java
core/src/main/java/org/apache/calcite/schema/impl/ViewTable.java
core/src/main/java/org/apache/calcite/schema/impl/ViewTableMacro.java
core/src/main/java/org/apache/calcite/schema/Schemas.java
core/src/main/java/org/apache/calcite/sql/validate/EmptyScope.java
core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorCatalogReader.java
core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java
core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorUtil.java
core/src/main/java/org/apache/calcite/tools/Frameworks.java

So should we go the opposite way and remove usage of CalciteSchema in
calcite core, outside of the JDBC adapter? It looks like some interfaces
would have to be extended to replace its use.

On Mon, Apr 9, 2018 at 3:23 PM, Julian Hyde  wrote:

> The policy that CalciteSchema should be private to Calcite hasn’t changed
> (it is a public interface only because… Java).
>
> It has never been private to the JDBC adapter. (The JDBC adapter readers
> from JDBC sources and is in org.apache.calcite.adapter.jdbc, whereas
> CalciteSchema is in org.apache.calcite.jdbc, which makes Calcite look like
> a JDBC database.)
>
> CalciteSchema is an abstract class. There are two concrete
> implementations: CachingCalciteSchema which remembers what it has read from
> the underlying source, and SimpleCalciteSchema that caches nothing. I know
> Drill (and I guess now Dremio) has a different policy, so how about adding
> an implementation of CalciteSchema that has that policy? I have been asking
> for this for a long time.
>
> If we did what you suggest, and make CalciteSchema (or a base interface of
> it) a public API then people will go and create implementations of that
> interface that will get broken all the time.
>
> Julian
>
>
> > On Apr 3, 2018, at 4:10 PM, Laurent Goujon  wrote:
> >
> > Hi,
> >
> > While working on rebasing our code on top of Calcite 1.16, I noticed
> that a
> > new method was introduced to SqlValidatorCatalogReader: CalciteSchema
> > `SqlValidatorCatalogReader#getRootSchema()` (not recently, it was added
> for
> > Calcite 1.12)
> >
> > My understanding of CalciteSchema was that this class is private and
> > specific to the JDBC adapter, but looking at the code base, I see it used
> > in lots of various places. Has the policy changed? if so, would it make
> > sense to change it to an interface or to relax the class? The current
> model
> > where all informations are stored in maps doesn't match our own model of
> > navigating our catalog.
> >
> > Cheers,
> >
> > Laurent
>
>

Re: Use of CalciteSchema across Calcite

[Laurent Goujon]

Sorry, I misinterpreted your last message. As for Dremio/Drill, the model
offered by CalciteSchema is not scalable as it requires all subschemas and
tables to be loaded, even if only one table is accessed: fetching all
metadata might be an expensive operation depending on the type and number
of sources, especially compared to the previous model where the validator
would not try to resolve a table by querying all subschemas in the path vs
directly asking the catalog reader for it.

I'm trying to see if it is possible to figure out a model where both
approaches can be satisfied.

Laurent

On Mon, Apr 9, 2018 at 11:07 PM, Laurent Goujon  wrote:

> That was my understanding, but it seems usage of CalciteSchema is
> prevalent outside the scope of the JDBC adapter:
>
> $ grep -rl import.*CalciteSchema core/src/main/java|grep -v calcite/jdbc
> core/src/main/java/org/apache/calcite/materialize/Lattice.java
> core/src/main/java/org/apache/calcite/materialize/
> MaterializationActor.java
> core/src/main/java/org/apache/calcite/materialize/
> MaterializationService.java
> core/src/main/java/org/apache/calcite/model/ModelHandler.java
> core/src/main/java/org/apache/calcite/plan/RelOptLattice.java
> core/src/main/java/org/apache/calcite/prepare/CalciteCatalogReader.java
> core/src/main/java/org/apache/calcite/prepare/CalciteMaterializer.java
> core/src/main/java/org/apache/calcite/prepare/CalcitePrepareImpl.java
> core/src/main/java/org/apache/calcite/prepare/PlannerImpl.java
> core/src/main/java/org/apache/calcite/prepare/Prepare.java
> core/src/main/java/org/apache/calcite/prepare/QueryableRelBuilder.java
> core/src/main/java/org/apache/calcite/prepare/RelOptTableImpl.java
> core/src/main/java/org/apache/calcite/rel/rules/
> AggregateStarTableRule.java
> core/src/main/java/org/apache/calcite/schema/impl/
> MaterializedViewTable.java
> core/src/main/java/org/apache/calcite/schema/impl/ViewTable.java
> core/src/main/java/org/apache/calcite/schema/impl/ViewTableMacro.java
> core/src/main/java/org/apache/calcite/schema/Schemas.java
> core/src/main/java/org/apache/calcite/sql/validate/EmptyScope.java
> core/src/main/java/org/apache/calcite/sql/validate/
> SqlValidatorCatalogReader.java
> core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java
> core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorUtil.java
> core/src/main/java/org/apache/calcite/tools/Frameworks.java
>
> So should we go the opposite way and remove usage of CalciteSchema in
> calcite core, outside of the JDBC adapter? It looks like some interfaces
> would have to be extended to replace its use.
>
> On Mon, Apr 9, 2018 at 3:23 PM, Julian Hyde  wrote:
>
>> The policy that CalciteSchema should be private to Calcite hasn’t changed
>> (it is a public interface only because… Java).
>>
>> It has never been private to the JDBC adapter. (The JDBC adapter readers
>> from JDBC sources and is in org.apache.calcite.adapter.jdbc, whereas
>> CalciteSchema is in org.apache.calcite.jdbc, which makes Calcite look like
>> a JDBC database.)
>>
>> CalciteSchema is an abstract class. There are two concrete
>> implementations: CachingCalciteSchema which remembers what it has read from
>> the underlying source, and SimpleCalciteSchema that caches nothing. I know
>> Drill (and I guess now Dremio) has a different policy, so how about adding
>> an implementation of CalciteSchema that has that policy? I have been asking
>> for this for a long time.
>>
>> If we did what you suggest, and make CalciteSchema (or a base interface
>> of it) a public API then people will go and create implementations of that
>> interface that will get broken all the time.
>>
>> Julian
>>
>>
>> > On Apr 3, 2018, at 4:10 PM, Laurent Goujon  wrote:
>> >
>> > Hi,
>> >
>> > While working on rebasing our code on top of Calcite 1.16, I noticed
>> that a
>> > new method was introduced to SqlValidatorCatalogReader: CalciteSchema
>> > `SqlValidatorCatalogReader#getRootSchema()` (not recently, it was
>> added for
>> > Calcite 1.12)
>> >
>> > My understanding of CalciteSchema was that this class is private and
>> > specific to the JDBC adapter, but looking at the code base, I see it
>> used
>> > in lots of various places. Has the policy changed? if so, would it make
>> > sense to change it to an interface or to relax the class? The current
>> model
>> > where all informations are stored in maps doesn't match our own model of
>> > navigating our catalog.
>> >
>> > Cheers,
>> >
>> > Laurent
>>
>>
>