[jira] [Created] (CALCITE-3343) Improve LogicalTableFunctionScan to match table function windowing
Rui Wang created CALCITE-3343:
-
Summary: Improve LogicalTableFunctionScan to match table function
windowing
Key: CALCITE-3343
URL: https://issues.apache.org/jira/browse/CALCITE-3343
Project: Calcite
Issue Type: Sub-task
Reporter: Rui Wang
Assignee: Rui Wang
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/logical/LogicalTableFunctionScan.java#L40
{code:java}
/**
* Creates a LogicalTableFunctionScan.
*
* @param clusterCluster that this relational expression belongs to
* @param inputs 0 or more relational inputs
* @param traitSet Trait set
* @param rexCallFunction invocation expression
* @param elementTypeElement type of the collection that will implement
* this table
* @param rowTypeRow type produced by function
* @param columnMappings Column mappings associated with this function
*/
public LogicalTableFunctionScan(
RelOptCluster cluster,
RelTraitSet traitSet,
List inputs,
RexNode rexCall,
Type elementType,
RelDataType rowType,
Set columnMappings) {
super(cluster, traitSet, inputs, rexCall, elementType, rowType,
columnMappings);
}
{code}
There might be a gap to make LogicalTableFunctionScan work for table function
windowing:
1. we need a RexInputRef parameter (converted from the DESCRIPTOR) to mark
which column is the watermarked column.
2. we need a List of RexNode(RexLiteral) for interval parameters.
The most straightforward approach while maintaining backward compatible, seems
is add another constructor with an additional List parameter. We could
save RexInputRef and internals (RexLiteral) to this parameter.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
Question about FilterIntoJoinRule
We are using calcite to query data in Kudu. We have implemented our own
sort rule that merges rows returned from kudu when a queries orders by
columns in the same order as the primary key of the kudu table.
For a query that does a left join, in order to get the planner to use my
sort rule I had to disable the FILTER_ON_JOIN rule and use
the DUMB_FILTER_ON_JOIN (which does not try to simplify outer joins). I
also had to modify the following code in FilterJoinRule.perform()
// Try to push down above filters. These are typically where clause
// filters. They can be pushed down if they are not on the NULL
// generating side.
boolean filterPushed = false;
if (RelOptUtil.classifyFilters(
join,
aboveFilters,
joinType,
needsPushInto(join),
!joinType.generatesNullsOnLeft(),
!joinType.generatesNullsOnRight(),
joinFilters,
leftFilters,
rightFilters)) {
filterPushed = true;
}
The above code sets the pushLeft and pushRight boolean flags only if the
join type does not generate nulls on the left or right side. I was
wondering if any one knows why this is required?
If a filter only references column present in the left or right table
shouldn't we always be able to push it down?
Re: [DISCUSS] Release apache-calcite-1.21.0 (release candidate 1)
I noticed the problem thanks to Andrei but I underestimated its importance. Sorry about that! On Thu, Sep 12, 2019 at 4:35 PM Julian Hyde wrote: > Yes, I screwed up. Everyone who voted screwed up. I should have voted ‘-1’ > because the hash of the artifacts I got from svn did not match the hash in > the email. Let’s all do better next time. > > Still, no harm done. We know now that we were voting on the correct > artifacts. We have a valid release. > > Julian > > > > On Sep 12, 2019, at 5:54 AM, Michael Mior wrote: > > > > +1 to everything Vladmir said. Thanks for the release Stamatis! I do > > agree that the checksum issue shouldn't be ignored although an update > > from the RM to the vote thread should be sufficient. Really, we rely > > on the email of the RM not being compromised anyway if we assume we > > can have a MITM between us and the hosted files. > > -- > > Michael Mior > > mm...@apache.org > > > > Le jeu. 12 sept. 2019 à 06:59, Vladimir Sitnikov > > a écrit : > >> > >> Stamatis, thanks for your work on this. > >> > >> Stamatis>The checksum hash that was communicated in the vote email was > wrong > >> Stamatis>given > >> Stamatis>that the correct one was send along with the artifacts and > people > >> used this > >> Stamatis>for the checks I assume there is no problem. > >> > >> I'm inclined that we should vote with -1 (or wait for RM to send the > >> updated checksum) when checksum in the mail does not match to the > checksum > >> of the archive. > >> > >> Well, it is OK, if release manager sends updates, however it should not > be > >> the case that actual checksum > >> differs from the one that was suggested in the vote mail. > >> > >> Different checksums might mean there's MITM attempt, and it sounds wrong > >> that we "kind of ignore it". > >> Even though I agree the impact in this case was quite low (e.g. I've > >> personally verified PGP signature and ensured it was SHA512 based), we > >> would probably want to refrain from repeating that practice. > >> > >> I would like to follow https://reproducible-builds.org/ to simplify > release > >> validation. > >> > >> Vladimir > >
[jira] [Created] (CALCITE-3342) sqltorelconverter adds extra fields with order by
Jess Balint created CALCITE-3342:
Summary: sqltorelconverter adds extra fields with order by
Key: CALCITE-3342
URL: https://issues.apache.org/jira/browse/CALCITE-3342
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.20.0
Reporter: Jess Balint
This is the input query:
{code:sql}
selectconcat(i.CONSTRAINT_SCHEMA, '_', i.CONSTRAINT_NAME) as INDEX_NAME,
ii.COLUMN_NAME,
ii.ORDINAL_POSITION,
case when i.CONSTRAINT_TYPE = 'PRIMARY KEY' then 'Y' else 'N' end as
PRIMARY_KEY
from
INFORMATION_SCHEMA.table_constraints i inner join
INFORMATION_SCHEMA.key_column_usage ii on
i.CONSTRAINT_SCHEMA = ii.CONSTRAINT_SCHEMA and
i.CONSTRAINT_NAME = ii.CONSTRAINT_NAME and
i.TABLE_SCHEMA = ii.TABLE_SCHEMA and
i.TABLE_NAME = ii.TABLE_NAME
where i.TABLE_SCHEMA = 'serf' and i.TABLE_NAME = 'aml_large_transfers'
and i.CONSTRAINT_TYPE in ('PRIMARY KEY', 'UNIQUE')
and ii.POSITION_IN_UNIQUE_CONSTRAINT is null
order by concat(i.CONSTRAINT_SCHEMA, '_', i.CONSTRAINT_NAME), ii.TABLE_SCHEMA,
ii.TABLE_NAME, ii.ORDINAL_POSITION
{code}
and SqlToRelConverter adds fields for the sorting but doesnt project them out:
{noformat}
LogicalSort(sort0=[$0], sort1=[$4], sort2=[$5], sort3=[$2], dir0=[ASC],
dir1=[ASC], dir2=[ASC], dir3=[ASC])
LogicalProject(INDEX_NAME=[||(||($1, '_'), $2)], COLUMN_NAME=[$12],
ORDINAL_POSITION=[$13], PRIMARY_KEY=[CASE(=($5, 'PRIMARY KEY'), 'Y', 'N')],
TABLE_SCHEMA=[$10], TABLE_NAME=[$11])
LogicalFilter(condition=[AND(=($3, 'serf'), =($4, 'aml_large_transfers'),
OR(=($5, 'PRIMARY KEY'), =($5, 'UNIQUE')), IS NULL($14))])
LogicalJoin(condition=[AND(=($1, $7), =($2, $8), =($3, $10), =($4,
$11))], joinType=[inner])
LogicalTableScan(table=[[INFORMATION_SCHEMA, TABLE_CONSTRAINTS]])
LogicalTableScan(table=[[INFORMATION_SCHEMA, KEY_COLUMN_USAGE]])
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (CALCITE-3341) Implement FINAL modifier functionality
Julian Feinauer created CALCITE-3341: Summary: Implement FINAL modifier functionality Key: CALCITE-3341 URL: https://issues.apache.org/jira/browse/CALCITE-3341 Project: Calcite Issue Type: Bug Reporter: Julian Feinauer And set default modifier to RUNNING that its compliant with Oracle implementation, see CALCITE-3302. -- This message was sent by Atlassian Jira (v8.3.2#803003)
Re: [DISCUSS] Release apache-calcite-1.21.0 (release candidate 1)
Yes, I screwed up. Everyone who voted screwed up. I should have voted ‘-1’ because the hash of the artifacts I got from svn did not match the hash in the email. Let’s all do better next time. Still, no harm done. We know now that we were voting on the correct artifacts. We have a valid release. Julian > On Sep 12, 2019, at 5:54 AM, Michael Mior wrote: > > +1 to everything Vladmir said. Thanks for the release Stamatis! I do > agree that the checksum issue shouldn't be ignored although an update > from the RM to the vote thread should be sufficient. Really, we rely > on the email of the RM not being compromised anyway if we assume we > can have a MITM between us and the hosted files. > -- > Michael Mior > mm...@apache.org > > Le jeu. 12 sept. 2019 à 06:59, Vladimir Sitnikov > a écrit : >> >> Stamatis, thanks for your work on this. >> >> Stamatis>The checksum hash that was communicated in the vote email was wrong >> Stamatis>given >> Stamatis>that the correct one was send along with the artifacts and people >> used this >> Stamatis>for the checks I assume there is no problem. >> >> I'm inclined that we should vote with -1 (or wait for RM to send the >> updated checksum) when checksum in the mail does not match to the checksum >> of the archive. >> >> Well, it is OK, if release manager sends updates, however it should not be >> the case that actual checksum >> differs from the one that was suggested in the vote mail. >> >> Different checksums might mean there's MITM attempt, and it sounds wrong >> that we "kind of ignore it". >> Even though I agree the impact in this case was quite low (e.g. I've >> personally verified PGP signature and ensured it was SHA512 based), we >> would probably want to refrain from repeating that practice. >> >> I would like to follow https://reproducible-builds.org/ to simplify release >> validation. >> >> Vladimir
Re: [DISCUSS] Release apache-calcite-1.21.0 (release candidate 1)
+1 to everything Vladmir said. Thanks for the release Stamatis! I do agree that the checksum issue shouldn't be ignored although an update from the RM to the vote thread should be sufficient. Really, we rely on the email of the RM not being compromised anyway if we assume we can have a MITM between us and the hosted files. -- Michael Mior mm...@apache.org Le jeu. 12 sept. 2019 à 06:59, Vladimir Sitnikov a écrit : > > Stamatis, thanks for your work on this. > > Stamatis>The checksum hash that was communicated in the vote email was wrong > Stamatis>given > Stamatis>that the correct one was send along with the artifacts and people > used this > Stamatis>for the checks I assume there is no problem. > > I'm inclined that we should vote with -1 (or wait for RM to send the > updated checksum) when checksum in the mail does not match to the checksum > of the archive. > > Well, it is OK, if release manager sends updates, however it should not be > the case that actual checksum > differs from the one that was suggested in the vote mail. > > Different checksums might mean there's MITM attempt, and it sounds wrong > that we "kind of ignore it". > Even though I agree the impact in this case was quite low (e.g. I've > personally verified PGP signature and ensured it was SHA512 based), we > would probably want to refrain from repeating that practice. > > I would like to follow https://reproducible-builds.org/ to simplify release > validation. > > Vladimir
Re: CALCITE-3158, Avatica: migrate build system to Gradle
Thanks everybody for helpful comments. At this point I'm confident the change is good, so I'm inclined to just commit it and cleanup glitches later if any. I'm inclined to remove pom.xml files from Git control as well. PS. There was Gradle-produced release candidate. You can find it under the subject of "[VOTE] Apache Calcite Avatica: Gradle-based release artifacts ready for review". Vladimir
Re: How does Calcite implement `Column Prune`?
Hi Han, I guess what you are looking for is RelFieldTrimmer [1] and it is not implemented as a transformation rule but as a separate optimization phase in this case. Best, Stamatis [1] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java On Thu, Sep 12, 2019 at 10:15 AM Han Mingcong wrote: > Hi all, > I’m learning query optimization recently. As known, Calcite uses a volcano > optimizer which is different from other optimizers such as SparkSQL’s > Catalyst. But I’m curious how does volcano optimizer implement rules like > `ColumnPruning` in Catalyst? Or which transformation rule does Calcite use > to achieve it? > For example, we have such a SQL: > > select a from t where b > 10; > > If the schema of t is `a int, b int, c int, …`, we only need two columns > ‘a’ and ‘b’ when scan table ‘t’. > > > Mingcong Han > 2019.9.12 >
Re: Is it possible to retrieve materialized view query rewrite from Calcite Plan
Hi Shubham, View based rewriting is performed during planning (check the call hierarchy of [1] for more details). If you obtain a plan (RelNode) after the planner then most likely rel2sql API should give you the right SQL string. On the other hand if you are just using sql2rel API then rewritings are not considered. Best, Stamatis [1] https://github.com/apache/calcite/blob/d3c718328d4c83fb24007c8349b31b420187/core/src/main/java/org/apache/calcite/plan/RelOptPlanner.java#L177 On Wed, Sep 11, 2019 at 3:42 PM Shubham Kumar wrote: > Hi Contributors, > > I have recently started to experiment with Apache Calcite's materialized > views and I needed a bit of help. > > I have explicitly defined materialized views which are stored in data > source by defining it in root.schema.materializations of model file. > > Now for a query which should use the Materialized View: > > Explain plan for "query" gives a plan which shows that the MV is being > utilized. > > However, I wanted the optimized rewrite SQL query if possible, not just the > plan. So far, I tried to use the rel2sql api but when I print > RelNode.toString(), it gives me a plan which involves scanning the raw > tables and hence the SQL generated by rel2sql is not the one which utilized > the materialized view. Any pointers to get the rewrite query. > > -- > Thanks & Regards > > Shubham Kumar >
Re: [DISCUSS] Release apache-calcite-1.21.0 (release candidate 1)
Stamatis, thanks for your work on this. Stamatis>The checksum hash that was communicated in the vote email was wrong Stamatis>given Stamatis>that the correct one was send along with the artifacts and people used this Stamatis>for the checks I assume there is no problem. I'm inclined that we should vote with -1 (or wait for RM to send the updated checksum) when checksum in the mail does not match to the checksum of the archive. Well, it is OK, if release manager sends updates, however it should not be the case that actual checksum differs from the one that was suggested in the vote mail. Different checksums might mean there's MITM attempt, and it sounds wrong that we "kind of ignore it". Even though I agree the impact in this case was quite low (e.g. I've personally verified PGP signature and ensured it was SHA512 based), we would probably want to refrain from repeating that practice. I would like to follow https://reproducible-builds.org/ to simplify release validation. Vladimir
Re: Is it possible that unquoted identifiers are not implicitly converted to upper case
Hi Feng,
You’re right, i get the same result with your suggestion, and either of the
following expressions is ok.
1. properties.put(CalciteConnectionProperty.LEX.camelName(), "MYSQL");
2. properties.put("lex", "MYSQL");
You’re familiar with Calcite :), and thanks for your help and kindness! ☺
Actually, we plan to develop a new feature, and i find Calcite is a great
option to meet our demand.
Thanks Calcite community, and hope two communities can build deeper connection.
P.S
Apache ShardingSphere(Incubator) is an open-source ecosystem consisted of a set
of distributed database middleware solutions.
| |
Juan Pan
|
|
panj...@apache.org
Juan Pan(Trista), Apache ShardingSphere
|
On 09/12/2019 17:32，Feng Zhu wrote：
Hi, Juan Pan:
You may find the logic in *UnregisterDriver#connect(String url, Properties
info)*
It just parses the key-value pairs in url's prefix and adds into the copy
of "info".
Therefore, I think the below config
*properties.put(CalciteConnecti**onProperty.LEX, Lex.MYSQL); *
should be aligned with your first usage:
*properties.put("lex", "MYSQL"); *
Juan Pan 于2019年9月12日周四 下午2:23写道：
Hi Feng,
Thanks for your promote reply. :)
Lex is just what i want. But when i tried to use it, i encountered another
problem.
The first usage is ok, but the second one doesn’t work. ThoughLex are used
in different methods, the result will be same, i think. Do i misunderstand?
Or is the second one wrong usage？
The first usage:
CONNECTION_URL = "jdbc:calcite:lex=MYSQL;model="
try (Connection connection = DriverManager.getConnection(CONNECTION_URL);
Statement statement = connection.createStatement()) {
// do some things
}
The second usage:
CONNECTION_URL = "jdbc:calcite:model="
Properties properties = new Properties();
properties.put(CalciteConnectionProperty.LEX, Lex.MYSQL);
try (Connection connection = DriverManager.getConnection(CONNECTION_URL,
properties);
Statement statement = connection.createStatement()) {
// do some things
}
Thanks again for your kindness, and waiting for u. :)
Regards,
Trista
| |
Juan Pan
|
|
panj...@apache.org
Juan Pan(Trista), Apache ShardingSphere
|
On 09/11/2019 20:23，Feng Zhu wrote：
Hi, JuanPan,
You can refer to Lex, which decides how identifiers are quoted, whether
they are converted to upper-case
org.apache.calcite.config.Lex
Regards
Juan Pan 于2019年9月11日周三 下午8:05写道：
Hi, all the committers and contributors,
This email is for your help.
I am now deep in Apache Calcite, and it’s great. Now, i want to know
whether it is possible that unquoted identifiers are not implicitly
converted to upper case?
For example, a SQL is `select name from test`, when it was executed, an
exception is thrown:
org.apache.calcite.sql.validate.SqlValidatorException: Object 'TEST' not
found within 'memory'; did you mean 'test’?
I wonder there is any setting that can make `name` and `test` recognized
correctly by Calcite without double quotes.
Thanks for your help.
Regards,
Trista
---
Email：panj...@apache.org
Juan Pan(Trista) Apache ShardingSphere
Re: Is it possible that unquoted identifiers are not implicitly converted to upper case
Hi, Juan Pan:
You may find the logic in *UnregisterDriver#connect(String url, Properties
info)*
It just parses the key-value pairs in url's prefix and adds into the copy
of "info".
Therefore, I think the below config
*properties.put(CalciteConnecti**onProperty.LEX, Lex.MYSQL); *
should be aligned with your first usage:
*properties.put("lex", "MYSQL"); *
Juan Pan 于2019年9月12日周四 下午2:23写道：
>
>
>
>
>
>
> Hi Feng,
>
>
>
>
> Thanks for your promote reply. :)
>
>
>
>
> Lex is just what i want. But when i tried to use it, i encountered another
> problem.
>
>
>
>
> The first usage is ok, but the second one doesn’t work. ThoughLex are used
> in different methods, the result will be same, i think. Do i misunderstand?
> Or is the second one wrong usage？
>
>
>
>
> The first usage:
>
>
>
>
> CONNECTION_URL = "jdbc:calcite:lex=MYSQL;model="
>
> try (Connection connection = DriverManager.getConnection(CONNECTION_URL);
>
> Statement statement = connection.createStatement()) {
>
> // do some things
>
> }
>
>
>
>
> The second usage:
>
>
>
>
> CONNECTION_URL = "jdbc:calcite:model="
>
> Properties properties = new Properties();
>
> properties.put(CalciteConnectionProperty.LEX, Lex.MYSQL);
>
> try (Connection connection = DriverManager.getConnection(CONNECTION_URL,
> properties);
>
> Statement statement = connection.createStatement()) {
>
> // do some things
>
> }
>
>
>
>
> Thanks again for your kindness, and waiting for u. :)
>
>
>
>
> Regards,
>
> Trista
>
>
>
>
>
> | |
> Juan Pan
> |
> |
> panj...@apache.org
> Juan Pan(Trista), Apache ShardingSphere
> |
>
>
> On 09/11/2019 20:23，Feng Zhu wrote：
> Hi, JuanPan,
> You can refer to Lex, which decides how identifiers are quoted, whether
> they are converted to upper-case
> org.apache.calcite.config.Lex
>
> Regards
>
> Juan Pan 于2019年9月11日周三 下午8:05写道：
>
>
>
> Hi, all the committers and contributors,
>
>
> This email is for your help.
>
>
> I am now deep in Apache Calcite, and it’s great. Now, i want to know
> whether it is possible that unquoted identifiers are not implicitly
> converted to upper case?
>
>
> For example, a SQL is `select name from test`, when it was executed, an
> exception is thrown:
>
>
> org.apache.calcite.sql.validate.SqlValidatorException: Object 'TEST' not
> found within 'memory'; did you mean 'test’?
>
>
> I wonder there is any setting that can make `name` and `test` recognized
> correctly by Calcite without double quotes.
>
>
> Thanks for your help.
>
>
> Regards,
> Trista
> ---
> Email：panj...@apache.org
> Juan Pan(Trista) Apache ShardingSphere
>
>
>
How does Calcite implement `Column Prune`?
Hi all, I’m learning query optimization recently. As known, Calcite uses a volcano optimizer which is different from other optimizers such as SparkSQL’s Catalyst. But I’m curious how does volcano optimizer implement rules like `ColumnPruning` in Catalyst? Or which transformation rule does Calcite use to achieve it? For example, we have such a SQL: select a from t where b > 10; If the schema of t is `a int, b int, c int, …`, we only need two columns ‘a’ and ‘b’ when scan table ‘t’. Mingcong Han 2019.9.12
Re: JavaScript Avatica client
Hi, Personally, I'd be happy to contribute it -- it's currently MIT licensed (as a matter of convention of my employer), but I'll double-check with my employer about whether there are any qualms about just contributing it to the project directly. For now, it's certainly fine with me if you link to it from the Calcite site. - Gabriel On Tue, Sep 10, 2019 at 7:24 PM Julian Hyde wrote: > I have to say… if you wish to contribute this, we would definitely > consider it. It would not be the first client in a non-JVM language: > Francis wrote the Go driver and now it is part of Calcite/Avatica. > > Julian > > > > On Sep 10, 2019, at 3:03 AM, Francis Chuang > wrote: > > > > Hi Gabriel, > > > > It's always very exciting to see new clients/drivers for Avatica that > are contributed by the community. > > > > We maintain a list of clients for Avatica on this page: > https://calcite.apache.org/avatica/docs/ > > > > Would it be okay to add your Javascript client to that page? > > > > Francis > > > > On 10/09/2019 5:45 pm, Gabriel Reid wrote: > >> Hi, > >> I just wanted to let anyone who's interested know that I've recently > been > >> working on a JavaScript (i.e. Node.js) client for Avatica. The current > >> status is that it does what I need it to do (i.e. it can execute a query > >> and return the results), and it could potentially be useful to others > (as > >> well as being built on further), although I'm sure that it doesn't cover > >> all use cases (e.g. currently only a subset of data types are handled). > >> The GitHub repo is here: https://github.com/waylayio/avatica-js > >> The npm entry is here: https://www.npmjs.com/package/avaticajs > >> - Gabriel > >
Re: Issues in exposing data via TableFunction vs TableMacro
Hi Julian, On Tue, Sep 10, 2019 at 5:19 PM Julian Feinauer < j.feina...@pragmaticminds.de> wrote: > > when going through the Code I just had another Idea. > Currently a TableFunction is executed as EnumerableTableFunctionScan which > gets generated from a LogicalTableFunctionScan by the Rule > EnumerableTableFunctionScanRule. > What if you just remove that Rule and add a custom Rule of yours which > translates it of a TableScan of your taste? > > That is indeed something I had thought about. In reference to your earlier question, yes, there are parameters needed for the TableFunction, so I think that the DrillTable approach wouldn't work in my case. I didn't see any easy way to alter rules outside of RelNode.register, but I assume that that is indeed possible, so I'll look further into this approach. Thanks for the advice. - Gabriel > Julian > > Am 10.09.19, 08:13 schrieb "Julian Feinauer" < > j.feina...@pragmaticminds.de>: > > Hi Gabriel, > > thats an interesting question for me too. > Do you need parameters for those "dynamic tables"? > If not you could do it similar to what Drill is doing and just > implement a Schema which always returns "true" if someone asks for a Table > and then returns a Table Implementation that you provide where you can hook > in later and add the functionality that you in fact need. This can then > also be used in optimization as you can then control your custom Table type. > Perhaps it helps to look at the DrillTable class in [1]. > > On a Side node I try to figure out what would be necessary to make > TableFunction wo also work with TranslatableTable. > Would you mind opening an issue in Jira for that? > > Julian > > [1] > https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillTable.java > > Am 10.09.19, 03:25 schrieb "Gabriel Reid" : > > Hi, > > I'm currently using a combination of TableFunctions and > TableMacros to > expose various dynamic (relatively unstructured) data sources via > Calcite. > The underlying data sources are such that data can only be > retrieved by > first specifying what you want (i.e. there is no catalog of all > data that > is available). > > I'm currently handling this by using a combination of > TableFunctions and > TableMacros. > > The issue that I'm running into comes when I want to implement > custom > planner rules for the underlying functionality. As far I as I can > see, it's > not possible to register planner rules based on a > TableFunctionImpl, > because a TableFunctionImpl only exposes a ScannableTable, so > there's no > chance to hook into RelOptNode.register. > > On the other hand, implementing a TableMacro does allow to return a > TranslatableTable, which then does allow intercepting the call to > RelOptNode.register to register rules. However, TableMacros > require that > all parameters are literals, and I'm providing timestamps, via > TIMESTAMPADD() and CURRENT_TIMESTAMP() calls, which then doesn't > work for > TableMacros (all parameters to a table macro need to be literals, > otherwise > query validation fails in Calcite). > > I'm wondering if I'm missing some built-in functionality which > would make > it possible to have a dynamic table function/macro that can also be > manipulated via custom planner rules. > > Options (which may or may not exist) that I can think of are: > * something that would/could visit all macro parameters ahead of > time and > resolve things like CURRENT_TIMESTAMP() to a literal, before > further query > validation occurs > * register rules somewhere outside of RelOptNode.register (e.g. > when the > schema is first created) > > Are there any currently-working options in Calcite that can help > me do what > I'm trying to do? And if there aren't and I would add such a thing > to > Calcite, are there any suggestions as to what the most appropriate > approach > would be (either one of the two options I listed above, or > something else)? > > Thanks, > > Gabriel > > > > >
Re: Is it possible that unquoted identifiers are not implicitly converted to upper case
Hi Feng,
Thanks for your promote reply. :)
Lex is just what i want. But when i tried to use it, i encountered another
problem.
The first usage is ok, but the second one doesn’t work. ThoughLex are used in
different methods, the result will be same, i think. Do i misunderstand? Or is
the second one wrong usage？
The first usage:
CONNECTION_URL = "jdbc:calcite:lex=MYSQL;model="
try (Connection connection = DriverManager.getConnection(CONNECTION_URL);
Statement statement = connection.createStatement()) {
// do some things
}
The second usage:
CONNECTION_URL = "jdbc:calcite:model="
Properties properties = new Properties();
properties.put(CalciteConnectionProperty.LEX, Lex.MYSQL);
try (Connection connection = DriverManager.getConnection(CONNECTION_URL,
properties);
Statement statement = connection.createStatement()) {
// do some things
}
Thanks again for your kindness, and waiting for u. :)
Regards,
Trista
| |
Juan Pan
|
|
panj...@apache.org
Juan Pan(Trista), Apache ShardingSphere
|
On 09/11/2019 20:23，Feng Zhu wrote：
Hi, JuanPan,
You can refer to Lex, which decides how identifiers are quoted, whether
they are converted to upper-case
org.apache.calcite.config.Lex
Regards
Juan Pan 于2019年9月11日周三 下午8:05写道：
Hi, all the committers and contributors,
This email is for your help.
I am now deep in Apache Calcite, and it’s great. Now, i want to know
whether it is possible that unquoted identifiers are not implicitly
converted to upper case?
For example, a SQL is `select name from test`, when it was executed, an
exception is thrown:
org.apache.calcite.sql.validate.SqlValidatorException: Object 'TEST' not
found within 'memory'; did you mean 'test’?
I wonder there is any setting that can make `name` and `test` recognized
correctly by Calcite without double quotes.
Thanks for your help.
Regards,
Trista
---
Email：panj...@apache.org
Juan Pan(Trista) Apache ShardingSphere
