Re: Druid-specific Calcite keywords

[Julian Hyde]

Those sound like good feature ideas. The way to get them into Calcite is to add 
new variables to the template file, provide empty defaults for the variables, 
and then override the variables in Druid’s parser. 

For an example, see the changes to core/src/main/codegen/config.fmpp in 
https://github.com/apache/calcite/commit/e8baf4f4720e07cdce1aa2baabf20042b9353bbb
 

 which added support for Postgres-style “::” cast operator in the Babel 
sub-parser but not the core parser.

Julian


> On Nov 5, 2021, at 8:19 AM, Gian Merlino  wrote:
> 
> Thanks for the note! Do you have any pointers to projects that do this in
> sort of a "best practices" way?
> 
> As to specifics: one thing I wanted to explore is adding keywords that do
> some of the same things as our query context parameters, so you don't have
> to set context parameters in order to get the behavior you want. (Sometimes
> people find it difficult to do that due to the abstractions that sit
> between them and the Druid API.) That's stuff like useApproximateTopN,
> which maybe could be "ORDER BY APPROXIMATE " or "ORDER BY 
>  APPROXIMATE".
> 
> I had also wanted to look at adding a PARTITION BY keyword that controls
> partitioning of query result sets.
> 
> On Thu, Nov 4, 2021 at 11:48 PM Julian Hyde  wrote:
> 
>> Some specifics would be useful. But in general, adding a new keyword
>> (reserved or non-reserved) will require changes to the paser. Calcite
>> allows (I won't say it makes it easy) for projects like Druid to
>> create a derived parser by building a parser from the same parser
>> template as Calcite's core parser but with different template
>> variables.
>> 
>> If the keyword is non-reserved, there is an additional grammar rule
>> that transforms the keyword token back into an identifier. It applies
>> in all contexts except the one where the keyword is specifically
>> needed by the grammar.  For example, the non-reserved keyword
>> BERNOULLI can only occur immediately after the keyword TABLESAMPLE. In
>> a location that expects an identifier (e.g. after FROM), BERNOULLI
>> will be converted into an identifier. Thus you can use BERNOULLI as a
>> table name.
>> 
>> Julian
>> 
>> On Thu, Nov 4, 2021 at 2:18 PM Gian Merlino  wrote:
>>> 
>>> Hey Druids,
>>> 
>>> I'm looking into how to add keywords to Druid's SQL dialect, and I wanted
>>> to ask if anyone has enough familiarity with Calcite to point at some
>> info
>>> about how to do that without needing to modify Calcite itself?
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
>> For additional commands, e-mail: dev-h...@druid.apache.org
>> 
>> 

Re: Druid-specific Calcite keywords

[Gian Merlino]

Thanks for the note! Do you have any pointers to projects that do this in
sort of a "best practices" way?

As to specifics: one thing I wanted to explore is adding keywords that do
some of the same things as our query context parameters, so you don't have
to set context parameters in order to get the behavior you want. (Sometimes
people find it difficult to do that due to the abstractions that sit
between them and the Druid API.) That's stuff like useApproximateTopN,
which maybe could be "ORDER BY APPROXIMATE " or "ORDER BY 
 APPROXIMATE".

I had also wanted to look at adding a PARTITION BY keyword that controls
partitioning of query result sets.

On Thu, Nov 4, 2021 at 11:48 PM Julian Hyde  wrote:

> Some specifics would be useful. But in general, adding a new keyword
> (reserved or non-reserved) will require changes to the paser. Calcite
> allows (I won't say it makes it easy) for projects like Druid to
> create a derived parser by building a parser from the same parser
> template as Calcite's core parser but with different template
> variables.
>
> If the keyword is non-reserved, there is an additional grammar rule
> that transforms the keyword token back into an identifier. It applies
> in all contexts except the one where the keyword is specifically
> needed by the grammar.  For example, the non-reserved keyword
> BERNOULLI can only occur immediately after the keyword TABLESAMPLE. In
> a location that expects an identifier (e.g. after FROM), BERNOULLI
> will be converted into an identifier. Thus you can use BERNOULLI as a
> table name.
>
> Julian
>
> On Thu, Nov 4, 2021 at 2:18 PM Gian Merlino  wrote:
> >
> > Hey Druids,
> >
> > I'm looking into how to add keywords to Druid's SQL dialect, and I wanted
> > to ask if anyone has enough familiarity with Calcite to point at some
> info
> > about how to do that without needing to modify Calcite itself?
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
>
>

Re: Druid-specific Calcite keywords

[Julian Hyde]

Some specifics would be useful. But in general, adding a new keyword
(reserved or non-reserved) will require changes to the paser. Calcite
allows (I won't say it makes it easy) for projects like Druid to
create a derived parser by building a parser from the same parser
template as Calcite's core parser but with different template
variables.

If the keyword is non-reserved, there is an additional grammar rule
that transforms the keyword token back into an identifier. It applies
in all contexts except the one where the keyword is specifically
needed by the grammar.  For example, the non-reserved keyword
BERNOULLI can only occur immediately after the keyword TABLESAMPLE. In
a location that expects an identifier (e.g. after FROM), BERNOULLI
will be converted into an identifier. Thus you can use BERNOULLI as a
table name.

Julian

On Thu, Nov 4, 2021 at 2:18 PM Gian Merlino  wrote:
>
> Hey Druids,
>
> I'm looking into how to add keywords to Druid's SQL dialect, and I wanted
> to ask if anyone has enough familiarity with Calcite to point at some info
> about how to do that without needing to modify Calcite itself?

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Druid-specific Calcite keywords

[Gian Merlino]

Hey Druids,

I'm looking into how to add keywords to Druid's SQL dialect, and I wanted
to ask if anyone has enough familiarity with Calcite to point at some info
about how to do that without needing to modify Calcite itself?