[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[
https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857366#comment-16857366
]
Danny Chan commented on CALCITE-2280:
-
This is actually what we have to do in Flink custom parser, for computation
engines like Flink, one big topic is the Hive compatibility, for example, if we
want to add "partition" grammar like
{code:java}
insert into T partition(col1=vl) select from S;
{code}
What we have to do is to copy the whole java cc parse block for insert
statement and add our custom parse logic.
We also have to add our SqlConformance Hive, but the SqlConformanceEnum is
enum, so i also have to hard code the conformance in the parser template.
> Liberal "babel" parser that accepts all SQL dialects
>
>
> Key: CALCITE-2280
> URL: https://issues.apache.org/jira/browse/CALCITE-2280
> Project: Calcite
> Issue Type: Bug
> Components: babel
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.17.0
>
>
> Create a parser that accepts all SQL dialects.
> It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery.
> If you have preferred dialects, please let us know in the comments section.
> (If you're willing to work on a particular dialect, even better!)
> We would do this in a new module, inheriting and extending the parser in the
> same way that the DDL parser in the "server" module does.
> This would be a messy and difficult project, because we would have to comply
> with the rules of each parser (and its set of built-in functions) rather than
> writing the rules as we would like them to be. That's why I would keep it out
> of the core parser. But it would also have large benefits.
> This would be new territory Calcite: as a tool for manipulating/understanding
> SQL, not (necessarily) for relational algebra or execution.
> Some possible uses:
> * analyze query lineage (what tables and columns are used in a query);
> * translate from one SQL dialect to another (using the JDBC adapter to
> generate SQL in the target dialect);
> * a "deep" compatibility mode (much more comprehensive than the current
> compatibility mode) where Calcite could pretend to be, say, Oracle;
> * SQL parser as a service: a REST call gives a SQL query, and returns a JSON
> or XML document with the parse tree.
> If you can think of interesting uses, please discuss in the comments.
> There are similarities with Uber's
> [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can
> collaborate, or make use of their test cases.
> We will need a lot of sample queries. If you are able to contribute sample
> queries for particular dialects, please discuss in the comments section. It
> would be good if the sample queries are based on a familiar schema (e.g.
> scott or foodmart) but we can be flexible about this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[ https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857007#comment-16857007 ] Julian Hyde commented on CALCITE-2280: -- I agree it would be nice if Babel parser could validate against a conformance. That said, I wouldn't let it dominate the design. In particular, Babel should work if people don't specify a conformance up-front, and that will remain its main mode of operation. If someone wants a parser specialized for a particular dialect, they can build one from Calcite's templates. They can even copy-paste code from Babel in order to make their parser. > Liberal "babel" parser that accepts all SQL dialects > > > Key: CALCITE-2280 > URL: https://issues.apache.org/jira/browse/CALCITE-2280 > Project: Calcite > Issue Type: Bug > Components: babel >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Fix For: 1.17.0 > > > Create a parser that accepts all SQL dialects. > It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery. > If you have preferred dialects, please let us know in the comments section. > (If you're willing to work on a particular dialect, even better!) > We would do this in a new module, inheriting and extending the parser in the > same way that the DDL parser in the "server" module does. > This would be a messy and difficult project, because we would have to comply > with the rules of each parser (and its set of built-in functions) rather than > writing the rules as we would like them to be. That's why I would keep it out > of the core parser. But it would also have large benefits. > This would be new territory Calcite: as a tool for manipulating/understanding > SQL, not (necessarily) for relational algebra or execution. > Some possible uses: > * analyze query lineage (what tables and columns are used in a query); > * translate from one SQL dialect to another (using the JDBC adapter to > generate SQL in the target dialect); > * a "deep" compatibility mode (much more comprehensive than the current > compatibility mode) where Calcite could pretend to be, say, Oracle; > * SQL parser as a service: a REST call gives a SQL query, and returns a JSON > or XML document with the parse tree. > If you can think of interesting uses, please discuss in the comments. > There are similarities with Uber's > [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can > collaborate, or make use of their test cases. > We will need a lot of sample queries. If you are able to contribute sample > queries for particular dialects, please discuss in the comments section. It > would be good if the sample queries are based on a familiar schema (e.g. > scott or foodmart) but we can be flexible about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[ https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856620#comment-16856620 ] Danny Chan commented on CALCITE-2280: - [~julianhyde] The Babel parser can mix-in all kinds of sql dialect, but it also needs to have the ability to specific the parser to one sql dialect, for the engines like Apache Flink or Apache Drill, supports all kinds of sql dialect is not that necessary and will confusing the user if the we give the parser grammar but not implement them in query execution. Maybe we should not always have a SqlConformance level, but make the dialect configurable and easily to extend will bring much user friendliness. > Liberal "babel" parser that accepts all SQL dialects > > > Key: CALCITE-2280 > URL: https://issues.apache.org/jira/browse/CALCITE-2280 > Project: Calcite > Issue Type: Bug > Components: babel >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Fix For: 1.17.0 > > > Create a parser that accepts all SQL dialects. > It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery. > If you have preferred dialects, please let us know in the comments section. > (If you're willing to work on a particular dialect, even better!) > We would do this in a new module, inheriting and extending the parser in the > same way that the DDL parser in the "server" module does. > This would be a messy and difficult project, because we would have to comply > with the rules of each parser (and its set of built-in functions) rather than > writing the rules as we would like them to be. That's why I would keep it out > of the core parser. But it would also have large benefits. > This would be new territory Calcite: as a tool for manipulating/understanding > SQL, not (necessarily) for relational algebra or execution. > Some possible uses: > * analyze query lineage (what tables and columns are used in a query); > * translate from one SQL dialect to another (using the JDBC adapter to > generate SQL in the target dialect); > * a "deep" compatibility mode (much more comprehensive than the current > compatibility mode) where Calcite could pretend to be, say, Oracle; > * SQL parser as a service: a REST call gives a SQL query, and returns a JSON > or XML document with the parse tree. > If you can think of interesting uses, please discuss in the comments. > There are similarities with Uber's > [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can > collaborate, or make use of their test cases. > We will need a lot of sample queries. If you are able to contribute sample > queries for particular dialects, please discuss in the comments section. It > would be good if the sample queries are based on a familiar schema (e.g. > scott or foodmart) but we can be flexible about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[ https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856539#comment-16856539 ] pingle wang commented on CALCITE-2280: -- Currently I am a developer of the hadoop ecosystem, because we are most familiar with Hive SQL, and there are already many Hive sql production tasks. Now we want to use Hive SQLas the standard (so existing SQL tasks can be run without changes), running in Spark, Flink, Druid or Drill. So I hope that Babel can be better close to Hive, thanks. Meitu mlink team > Liberal "babel" parser that accepts all SQL dialects > > > Key: CALCITE-2280 > URL: https://issues.apache.org/jira/browse/CALCITE-2280 > Project: Calcite > Issue Type: Bug > Components: babel >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Fix For: 1.17.0 > > > Create a parser that accepts all SQL dialects. > It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery. > If you have preferred dialects, please let us know in the comments section. > (If you're willing to work on a particular dialect, even better!) > We would do this in a new module, inheriting and extending the parser in the > same way that the DDL parser in the "server" module does. > This would be a messy and difficult project, because we would have to comply > with the rules of each parser (and its set of built-in functions) rather than > writing the rules as we would like them to be. That's why I would keep it out > of the core parser. But it would also have large benefits. > This would be new territory Calcite: as a tool for manipulating/understanding > SQL, not (necessarily) for relational algebra or execution. > Some possible uses: > * analyze query lineage (what tables and columns are used in a query); > * translate from one SQL dialect to another (using the JDBC adapter to > generate SQL in the target dialect); > * a "deep" compatibility mode (much more comprehensive than the current > compatibility mode) where Calcite could pretend to be, say, Oracle; > * SQL parser as a service: a REST call gives a SQL query, and returns a JSON > or XML document with the parse tree. > If you can think of interesting uses, please discuss in the comments. > There are similarities with Uber's > [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can > collaborate, or make use of their test cases. > We will need a lot of sample queries. If you are able to contribute sample > queries for particular dialects, please discuss in the comments section. It > would be good if the sample queries are based on a familiar schema (e.g. > scott or foodmart) but we can be flexible about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[
https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650969#comment-16650969
]
Julian Hyde commented on CALCITE-2280:
--
We can definitely add HIVE as a conformance level. As for adding Hive QL
features to Calcite's core SQL parser, it depends on the particular feature:
* If a feature is in the SQL standard, then we should add it.
* If a feature is generally useful, and occurs in several databases, then we
should add it, enabled by a conformance flag. ({{ORDER BY ordinal}} is an
example of such a feature.)
* If a feature is peculiar to just one database, and does not offer a
significant extra functionality, or makes the parser ambiguous or more complex
for other dialects, then we probably will not add it to Calcite's dialect.
Note that Hive QL is now called Hive SQL, because Hive is trying to move closer
to standard SQL over time. I don't think we should be adding to Calcite support
for features that the Hive team no longer recommends that people use in Hive.
For such features, Babel is probably a better home.
> Liberal "babel" parser that accepts all SQL dialects
>
>
> Key: CALCITE-2280
> URL: https://issues.apache.org/jira/browse/CALCITE-2280
> Project: Calcite
> Issue Type: Bug
> Components: babel
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.17.0
>
>
> Create a parser that accepts all SQL dialects.
> It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery.
> If you have preferred dialects, please let us know in the comments section.
> (If you're willing to work on a particular dialect, even better!)
> We would do this in a new module, inheriting and extending the parser in the
> same way that the DDL parser in the "server" module does.
> This would be a messy and difficult project, because we would have to comply
> with the rules of each parser (and its set of built-in functions) rather than
> writing the rules as we would like them to be. That's why I would keep it out
> of the core parser. But it would also have large benefits.
> This would be new territory Calcite: as a tool for manipulating/understanding
> SQL, not (necessarily) for relational algebra or execution.
> Some possible uses:
> * analyze query lineage (what tables and columns are used in a query);
> * translate from one SQL dialect to another (using the JDBC adapter to
> generate SQL in the target dialect);
> * a "deep" compatibility mode (much more comprehensive than the current
> compatibility mode) where Calcite could pretend to be, say, Oracle;
> * SQL parser as a service: a REST call gives a SQL query, and returns a JSON
> or XML document with the parse tree.
> If you can think of interesting uses, please discuss in the comments.
> There are similarities with Uber's
> [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can
> collaborate, or make use of their test cases.
> We will need a lot of sample queries. If you are able to contribute sample
> queries for particular dialects, please discuss in the comments section. It
> would be good if the sample queries are based on a familiar schema (e.g.
> scott or foodmart) but we can be flexible about this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[ https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650899#comment-16650899 ] Xuefu Zhang commented on CALCITE-2280: -- Hi [~julianhyde] With my limited understanding of this change, I assume we have put a boiler place for extending Calcite grammar to accommodate any specialties in any SQL dialects. This is great! On the other hand, I'm curious about whether we can provide, or have a plan to do so, a conformance level for Hive QL as a whole, similar to MySQL, given the popularity of Hive QL in the big data world. Could you shed some light on this? Thanks. > Liberal "babel" parser that accepts all SQL dialects > > > Key: CALCITE-2280 > URL: https://issues.apache.org/jira/browse/CALCITE-2280 > Project: Calcite > Issue Type: Bug > Components: babel >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Fix For: 1.17.0 > > > Create a parser that accepts all SQL dialects. > It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery. > If you have preferred dialects, please let us know in the comments section. > (If you're willing to work on a particular dialect, even better!) > We would do this in a new module, inheriting and extending the parser in the > same way that the DDL parser in the "server" module does. > This would be a messy and difficult project, because we would have to comply > with the rules of each parser (and its set of built-in functions) rather than > writing the rules as we would like them to be. That's why I would keep it out > of the core parser. But it would also have large benefits. > This would be new territory Calcite: as a tool for manipulating/understanding > SQL, not (necessarily) for relational algebra or execution. > Some possible uses: > * analyze query lineage (what tables and columns are used in a query); > * translate from one SQL dialect to another (using the JDBC adapter to > generate SQL in the target dialect); > * a "deep" compatibility mode (much more comprehensive than the current > compatibility mode) where Calcite could pretend to be, say, Oracle; > * SQL parser as a service: a REST call gives a SQL query, and returns a JSON > or XML document with the parse tree. > If you can think of interesting uses, please discuss in the comments. > There are similarities with Uber's > [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can > collaborate, or make use of their test cases. > We will need a lot of sample queries. If you are able to contribute sample > queries for particular dialects, please discuss in the comments section. It > would be good if the sample queries are based on a familiar schema (e.g. > scott or foodmart) but we can be flexible about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[ https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520767#comment-16520767 ] Enrico Olivelli commented on CALCITE-2280: -- I guess we should wait for Calcite 1.18 for this great feature. Not really a problem, we will have more time to give feedback once this patch is merged to master. Thank you again Julian > Liberal "babel" parser that accepts all SQL dialects > > > Key: CALCITE-2280 > URL: https://issues.apache.org/jira/browse/CALCITE-2280 > Project: Calcite > Issue Type: Bug > Components: babel >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Fix For: 1.17.0 > > > Create a parser that accepts all SQL dialects. > It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery. > If you have preferred dialects, please let us know in the comments section. > (If you're willing to work on a particular dialect, even better!) > We would do this in a new module, inheriting and extending the parser in the > same way that the DDL parser in the "server" module does. > This would be a messy and difficult project, because we would have to comply > with the rules of each parser (and its set of built-in functions) rather than > writing the rules as we would like them to be. That's why I would keep it out > of the core parser. But it would also have large benefits. > This would be new territory Calcite: as a tool for manipulating/understanding > SQL, not (necessarily) for relational algebra or execution. > Some possible uses: > * analyze query lineage (what tables and columns are used in a query); > * translate from one SQL dialect to another (using the JDBC adapter to > generate SQL in the target dialect); > * a "deep" compatibility mode (much more comprehensive than the current > compatibility mode) where Calcite could pretend to be, say, Oracle; > * SQL parser as a service: a REST call gives a SQL query, and returns a JSON > or XML document with the parse tree. > If you can think of interesting uses, please discuss in the comments. > There are similarities with Uber's > [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can > collaborate, or make use of their test cases. > We will need a lot of sample queries. If you are able to contribute sample > queries for particular dialects, please discuss in the comments section. It > would be good if the sample queries are based on a familiar schema (e.g. > scott or foodmart) but we can be flexible about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[ https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520703#comment-16520703 ] Julian Hyde commented on CALCITE-2280: -- I have implemented CALCITE-2280 based on an early version of CALCITE-2259 so I cannot commit 2280 until 2259 is completed. > Liberal "babel" parser that accepts all SQL dialects > > > Key: CALCITE-2280 > URL: https://issues.apache.org/jira/browse/CALCITE-2280 > Project: Calcite > Issue Type: Bug > Components: babel >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Fix For: 1.17.0 > > > Create a parser that accepts all SQL dialects. > It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery. > If you have preferred dialects, please let us know in the comments section. > (If you're willing to work on a particular dialect, even better!) > We would do this in a new module, inheriting and extending the parser in the > same way that the DDL parser in the "server" module does. > This would be a messy and difficult project, because we would have to comply > with the rules of each parser (and its set of built-in functions) rather than > writing the rules as we would like them to be. That's why I would keep it out > of the core parser. But it would also have large benefits. > This would be new territory Calcite: as a tool for manipulating/understanding > SQL, not (necessarily) for relational algebra or execution. > Some possible uses: > * analyze query lineage (what tables and columns are used in a query); > * translate from one SQL dialect to another (using the JDBC adapter to > generate SQL in the target dialect); > * a "deep" compatibility mode (much more comprehensive than the current > compatibility mode) where Calcite could pretend to be, say, Oracle; > * SQL parser as a service: a REST call gives a SQL query, and returns a JSON > or XML document with the parse tree. > If you can think of interesting uses, please discuss in the comments. > There are similarities with Uber's > [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can > collaborate, or make use of their test cases. > We will need a lot of sample queries. If you are able to contribute sample > queries for particular dialects, please discuss in the comments section. It > would be good if the sample queries are based on a familiar schema (e.g. > scott or foodmart) but we can be flexible about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[ https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16467962#comment-16467962 ] Julian Hyde commented on CALCITE-2280: -- I've pushed my dev branch as https://github.com/julianhyde/calcite/tree/2280-babel. (It's ready to go, but I can't merge to master until CALCITE-2259 is finished.) It also fixes CALCITE-2304, which illustrates how to add a small syntax to Babel and its tests. Now I've set the ball rolling, it's up to you. Please contribute more tests and syntax features! > Liberal "babel" parser that accepts all SQL dialects > > > Key: CALCITE-2280 > URL: https://issues.apache.org/jira/browse/CALCITE-2280 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > > Create a parser that accepts all SQL dialects. > It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery. > If you have preferred dialects, please let us know in the comments section. > (If you're willing to work on a particular dialect, even better!) > We would do this in a new module, inheriting and extending the parser in the > same way that the DDL parser in the "server" module does. > This would be a messy and difficult project, because we would have to comply > with the rules of each parser (and its set of built-in functions) rather than > writing the rules as we would like them to be. That's why I would keep it out > of the core parser. But it would also have large benefits. > This would be new territory Calcite: as a tool for manipulating/understanding > SQL, not (necessarily) for relational algebra or execution. > Some possible uses: > * analyze query lineage (what tables and columns are used in a query); > * translate from one SQL dialect to another (using the JDBC adapter to > generate SQL in the target dialect); > * a "deep" compatibility mode (much more comprehensive than the current > compatibility mode) where Calcite could pretend to be, say, Oracle; > * SQL parser as a service: a REST call gives a SQL query, and returns a JSON > or XML document with the parse tree. > If you can think of interesting uses, please discuss in the comments. > There are similarities with Uber's > [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can > collaborate, or make use of their test cases. > We will need a lot of sample queries. If you are able to contribute sample > queries for particular dialects, please discuss in the comments section. It > would be good if the sample queries are based on a familiar schema (e.g. > scott or foodmart) but we can be flexible about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[ https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466492#comment-16466492 ] Julian Hyde commented on CALCITE-2280: -- [~SashaO], Yes, those queries should definitely work (for some dialects at least -- for other dialects they may present a conflict, or require quoting). It's important that we handle these "everyday" kinds of differences. So, yes, please add them as test cases. > Liberal "babel" parser that accepts all SQL dialects > > > Key: CALCITE-2280 > URL: https://issues.apache.org/jira/browse/CALCITE-2280 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > > Create a parser that accepts all SQL dialects. > It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery. > If you have preferred dialects, please let us know in the comments section. > (If you're willing to work on a particular dialect, even better!) > We would do this in a new module, inheriting and extending the parser in the > same way that the DDL parser in the "server" module does. > This would be a messy and difficult project, because we would have to comply > with the rules of each parser (and its set of built-in functions) rather than > writing the rules as we would like them to be. That's why I would keep it out > of the core parser. But it would also have large benefits. > This would be new territory Calcite: as a tool for manipulating/understanding > SQL, not (necessarily) for relational algebra or execution. > Some possible uses: > * analyze query lineage (what tables and columns are used in a query); > * translate from one SQL dialect to another (using the JDBC adapter to > generate SQL in the target dialect); > * a "deep" compatibility mode (much more comprehensive than the current > compatibility mode) where Calcite could pretend to be, say, Oracle; > * SQL parser as a service: a REST call gives a SQL query, and returns a JSON > or XML document with the parse tree. > If you can think of interesting uses, please discuss in the comments. > There are similarities with Uber's > [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can > collaborate, or make use of their test cases. > We will need a lot of sample queries. If you are able to contribute sample > queries for particular dialects, please discuss in the comments section. It > would be good if the sample queries are based on a familiar schema (e.g. > scott or foodmart) but we can be flexible about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[ https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466491#comment-16466491 ] Erica Ehrhardt commented on CALCITE-2280: - Also, +1 on [~SashaO]'s comment. BigQuery, for example, allows quite a few common words (such as "date" and "time") as unquoted identifiers, which is actually pretty convenient, so my hacked-up copy of the Calcite grammar has to un-reserve a whole bunch of keywords. > Liberal "babel" parser that accepts all SQL dialects > > > Key: CALCITE-2280 > URL: https://issues.apache.org/jira/browse/CALCITE-2280 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > > Create a parser that accepts all SQL dialects. > It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery. > If you have preferred dialects, please let us know in the comments section. > (If you're willing to work on a particular dialect, even better!) > We would do this in a new module, inheriting and extending the parser in the > same way that the DDL parser in the "server" module does. > This would be a messy and difficult project, because we would have to comply > with the rules of each parser (and its set of built-in functions) rather than > writing the rules as we would like them to be. That's why I would keep it out > of the core parser. But it would also have large benefits. > This would be new territory Calcite: as a tool for manipulating/understanding > SQL, not (necessarily) for relational algebra or execution. > Some possible uses: > * analyze query lineage (what tables and columns are used in a query); > * translate from one SQL dialect to another (using the JDBC adapter to > generate SQL in the target dialect); > * a "deep" compatibility mode (much more comprehensive than the current > compatibility mode) where Calcite could pretend to be, say, Oracle; > * SQL parser as a service: a REST call gives a SQL query, and returns a JSON > or XML document with the parse tree. > If you can think of interesting uses, please discuss in the comments. > There are similarities with Uber's > [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can > collaborate, or make use of their test cases. > We will need a lot of sample queries. If you are able to contribute sample > queries for particular dialects, please discuss in the comments section. It > would be good if the sample queries are based on a familiar schema (e.g. > scott or foodmart) but we can be flexible about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[ https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466489#comment-16466489 ] Erica Ehrhardt commented on CALCITE-2280: - I'm working on approval to release some of our queries. Thanks for the Quidem format overview, [~julianhyde]; that looks fairly convenient. > Liberal "babel" parser that accepts all SQL dialects > > > Key: CALCITE-2280 > URL: https://issues.apache.org/jira/browse/CALCITE-2280 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > > Create a parser that accepts all SQL dialects. > It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery. > If you have preferred dialects, please let us know in the comments section. > (If you're willing to work on a particular dialect, even better!) > We would do this in a new module, inheriting and extending the parser in the > same way that the DDL parser in the "server" module does. > This would be a messy and difficult project, because we would have to comply > with the rules of each parser (and its set of built-in functions) rather than > writing the rules as we would like them to be. That's why I would keep it out > of the core parser. But it would also have large benefits. > This would be new territory Calcite: as a tool for manipulating/understanding > SQL, not (necessarily) for relational algebra or execution. > Some possible uses: > * analyze query lineage (what tables and columns are used in a query); > * translate from one SQL dialect to another (using the JDBC adapter to > generate SQL in the target dialect); > * a "deep" compatibility mode (much more comprehensive than the current > compatibility mode) where Calcite could pretend to be, say, Oracle; > * SQL parser as a service: a REST call gives a SQL query, and returns a JSON > or XML document with the parse tree. > If you can think of interesting uses, please discuss in the comments. > There are similarities with Uber's > [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can > collaborate, or make use of their test cases. > We will need a lot of sample queries. If you are able to contribute sample > queries for particular dialects, please discuss in the comments section. It > would be good if the sample queries are based on a familiar schema (e.g. > scott or foodmart) but we can be flexible about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[ https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466487#comment-16466487 ] Sasha Ovsankin commented on CALCITE-2280: - [~julianhyde] one class of errors we are seeing quite a bit is the use of SQL reserved keywords as identifiers, such as "date", "returns", "nulls", "period", "position". While these may not be valid "standard" SQL, they actually happen in real queries quite a bit because the respective platform doesn't mind. Rejecting them would mean that queries that used to work, now don't. What is your take on these? How "liberal" should the liberal parser be with them? > Liberal "babel" parser that accepts all SQL dialects > > > Key: CALCITE-2280 > URL: https://issues.apache.org/jira/browse/CALCITE-2280 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > > Create a parser that accepts all SQL dialects. > It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery. > If you have preferred dialects, please let us know in the comments section. > (If you're willing to work on a particular dialect, even better!) > We would do this in a new module, inheriting and extending the parser in the > same way that the DDL parser in the "server" module does. > This would be a messy and difficult project, because we would have to comply > with the rules of each parser (and its set of built-in functions) rather than > writing the rules as we would like them to be. That's why I would keep it out > of the core parser. But it would also have large benefits. > This would be new territory Calcite: as a tool for manipulating/understanding > SQL, not (necessarily) for relational algebra or execution. > Some possible uses: > * analyze query lineage (what tables and columns are used in a query); > * translate from one SQL dialect to another (using the JDBC adapter to > generate SQL in the target dialect); > * a "deep" compatibility mode (much more comprehensive than the current > compatibility mode) where Calcite could pretend to be, say, Oracle; > * SQL parser as a service: a REST call gives a SQL query, and returns a JSON > or XML document with the parse tree. > If you can think of interesting uses, please discuss in the comments. > There are similarities with Uber's > [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can > collaborate, or make use of their test cases. > We will need a lot of sample queries. If you are able to contribute sample > queries for particular dialects, please discuss in the comments section. It > would be good if the sample queries are based on a familiar schema (e.g. > scott or foodmart) but we can be flexible about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[
https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16464922#comment-16464922
]
Julian Hyde commented on CALCITE-2280:
--
[~eolivelli], [~eee] (and anyone else interested in contributing queries),
content is more important than format, but a fragment of a
[Quidem|https://github.com/julianhyde/quidem] script should be easy for you to
write. I just finished adding custom commands to Quidem (see
[[QUIDEM-18|https://github.com/julianhyde/quidem/issues/18]]) so now we can add
a custom {{explain-validated-on}} command, and execute scripts that look like
this:
{code}
# Test CONNECT BY (Oracle only)
SELECT *
FROM emp
START WITH mgr IS NULL
CONNECT BY empno = PRIOR mgr;
select(...)
!explain-validated-on oracle
# WITH RECURSIVE (Oracle, MySQL 8 onwards)
WITH RECURSIVE t(n) AS (
VALUES (1)
UNION ALL
SELECT n+1 FROM t WHERE n < 100
)
SELECT sum(n) FROM t;
select(...)
!explain-validated-on mysql8+ oracle
{code}
The "select(...)" before each '!' line is the validated and annotated parse
tree. Don't worry about that line. The Calcite developer will replace it with
the correct output.
Please contribute queries like that by attaching .iq files to this JIRA case.
The more the better!
> Liberal "babel" parser that accepts all SQL dialects
>
>
> Key: CALCITE-2280
> URL: https://issues.apache.org/jira/browse/CALCITE-2280
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
>
> Create a parser that accepts all SQL dialects.
> It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery.
> If you have preferred dialects, please let us know in the comments section.
> (If you're willing to work on a particular dialect, even better!)
> We would do this in a new module, inheriting and extending the parser in the
> same way that the DDL parser in the "server" module does.
> This would be a messy and difficult project, because we would have to comply
> with the rules of each parser (and its set of built-in functions) rather than
> writing the rules as we would like them to be. That's why I would keep it out
> of the core parser. But it would also have large benefits.
> This would be new territory Calcite: as a tool for manipulating/understanding
> SQL, not (necessarily) for relational algebra or execution.
> Some possible uses:
> * analyze query lineage (what tables and columns are used in a query);
> * translate from one SQL dialect to another (using the JDBC adapter to
> generate SQL in the target dialect);
> * a "deep" compatibility mode (much more comprehensive than the current
> compatibility mode) where Calcite could pretend to be, say, Oracle;
> * SQL parser as a service: a REST call gives a SQL query, and returns a JSON
> or XML document with the parse tree.
> If you can think of interesting uses, please discuss in the comments.
> There are similarities with Uber's
> [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can
> collaborate, or make use of their test cases.
> We will need a lot of sample queries. If you are able to contribute sample
> queries for particular dialects, please discuss in the comments section. It
> would be good if the sample queries are based on a familiar schema (e.g.
> scott or foodmart) but we can be flexible about this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[ https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461549#comment-16461549 ] Enrico Olivelli commented on CALCITE-2280: -- Great work! I have to face several problems due to some incompatibility with Mysql list of reserved words. Like 'timestamp', 'value' as column name and 'default' as schema name. I will be happy to contribute queries and eventually a more Mysql friendly parser > Liberal "babel" parser that accepts all SQL dialects > > > Key: CALCITE-2280 > URL: https://issues.apache.org/jira/browse/CALCITE-2280 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > > Create a parser that accepts all SQL dialects. > It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery. > If you have preferred dialects, please let us know in the comments section. > (If you're willing to work on a particular dialect, even better!) > We would do this in a new module, inheriting and extending the parser in the > same way that the DDL parser in the "server" module does. > This would be a messy and difficult project, because we would have to comply > with the rules of each parser (and its set of built-in functions) rather than > writing the rules as we would like them to be. That's why I would keep it out > of the core parser. But it would also have large benefits. > This would be new territory Calcite: as a tool for manipulating/understanding > SQL, not (necessarily) for relational algebra or execution. > Some possible uses: > * analyze query lineage (what tables and columns are used in a query); > * translate from one SQL dialect to another (using the JDBC adapter to > generate SQL in the target dialect); > * a "deep" compatibility mode (much more comprehensive than the current > compatibility mode) where Calcite could pretend to be, say, Oracle; > * SQL parser as a service: a REST call gives a SQL query, and returns a JSON > or XML document with the parse tree. > If you can think of interesting uses, please discuss in the comments. > There are similarities with Uber's > [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can > collaborate, or make use of their test cases. > We will need a lot of sample queries. If you are able to contribute sample > queries for particular dialects, please discuss in the comments section. It > would be good if the sample queries are based on a familiar schema (e.g. > scott or foodmart) but we can be flexible about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
