Re: Re: [DISCUSS] Refactor how planner rules are parameterized
I have force-pushed to https://github.com/apache/calcite/pull/2024.
There are now some commits to be merged before 1.24 (hopefully today
or tomorrow) and some to be merged after 1.24.
Can someone give it a final review?
Chunwei,
Please, let's do the 1.24 release sooner rather than later. The part
of this change that has to occur after 1.24 is large (204 files, 9k
lines added, 6k lines removed) and so is susceptible to bit-rot. I
agreed to Stamatis' proposal to do this in two phases on the
assumption that the release was not far off.
Smaller releases are easier for the release manager, too. :)
Julian
On Mon, Jul 6, 2020 at 1:40 AM Chunwei Lei wrote:
>
> If I am right, I am the release manager of 1.24.
>
> I suggest releasing 1.24 a bit later(at the end of this month)
> since 1.23 was released not long ago(on 2020.5.23).
>
>
> Best,
> Chunwei
>
>
> On Fri, Jul 3, 2020 at 10:54 PM Michael Mior wrote:
>
> > I'll just add for the relatively common case where a rule is matching
> > a node with a specific node as a child, and so on that we could easily
> > add a less verbose API to make this part of the rule definition
> > equally concise.
> >
> > --
> > Michael Mior
> > mm...@apache.org
> >
> > Le dim. 14 juin 2020 à 15:36, Haisheng Yuan a écrit :
> > >
> > > Hi Julian,
> > >
> > > Thanks for working on this.
> > >
> > > We haven't reached a consensus yet.
> > >
> > > Frankly speaking, I agree with what Stamatis said earlier. Flexibility
> > doesn't come with no cost. Admittedly, with this patch, any rule
> > constructor refactoring won't need to deprecate old constructors and break
> > backward compatibility, however, it also makes the rule definition much
> > more verbose, less readable and understandable. IMHO, it does more harm
> > than good.
> > >
> > > Let's see how CassandraFilterRule becomes before and after the change.
> > >
> > > Before this change:
> > >
> > > private static class CassandraFilterRule extends RelOptRule {
> > > private static final CassandraFilterRule INSTANCE = new
> > CassandraFilterRule();
> > >
> > > private CassandraFilterRule() {
> > > super(operand(LogicalFilter.class,
> > operand(CassandraTableScan.class, none())),
> > > "CassandraFilterRule");
> > > }
> > > }
> > >
> > > After this change:
> > >
> > > private static class CassandraFilterRule
> > > extends RelOptNewRule {
> > > private static final CassandraFilterRule INSTANCE =
> > > Config.EMPTY
> > > .withOperandSupplier(b0 ->
> > > b0.operand(LogicalFilter.class)
> > > .oneInput(b1 -> b1.operand(CassandraTableScan.class)
> > > .noInputs()))
> > > .as(Config.class)
> > > .toRule();
> > >
> > > /** Creates a CassandraFilterRule. */
> > > protected CassandraFilterRule(Config config) {
> > > super(config);
> > > }
> > >
> > > /** Rule configuration. */
> > > public interface Config extends RelOptNewRule.Config {
> > > @Override default CassandraFilterRule toRule() {
> > > return new CassandraFilterRule(this);
> > > }
> > > }
> > > }
> > >
> > >
> > > Intuitively, if we want to check what does the rule generally match or
> > how it is defined, we just check the constructor. Now we checkout the
> > constructor, only config is there, go to Config, there is still nothing
> > interesting, we have to go to the INSTANCE. What is the difference with
> > just using operand and optionConfig as the rule constructor?
> > >
> > >protected CassandraFilterRule(RelOptRuleOperand operand, Config
> > config) {
> > > super(operand, config);
> > >}
> > >
> > > Or even simply replace Config with int, with each bit represent an
> > option, if all of them are boolean options.
> > >
> > > Nothing is more flexible than just using RelOptRuleOperand as the
> > parameter, just like the base class RelOptRule does. But do we want it?
> > >
> > > At the same time, with the new approach, it is now legit to create the
> > following instance:
> > >
> > > private static final CassandraFilterRule INSTANCE2 =
> > > Config.EMPTY
> > > .withOperandSupplier(b0 ->
> > > b0.operand(LogicalProject.class) // Even the is
> > intended to match Filter, but it compiles
> > > .oneInput(b1 -> b1.operand(CassandraTableScan.class)
> > > .noInputs()))
> > > .as(Config.class)
> > > .toRule();
> > >
> > > Before we continue to the discussion and code review, we need to go back
> > to the original problem, is it a real problem that is facing us? Is there
> > any real demand or just artificial demand? How about we conduct a Calcite
> > user survey to see how Calcite devs and users think? Then let's see how to
> > move forward.
> > >
> > > [+1] Hell yeah, the new approach is awesome, let's go with it.
> > > [+0] Meh, I am OK with current approach, I don't see
Re: [DISCUSS] Make RexNode serializable
BTW, if use 'externalize', what is the opposite side of it? Is it `internalize` (which doesn't sound right)? Or call it "de-externalize"? -Rui On Wed, Jul 8, 2020 at 4:02 PM Rui Wang wrote: > O got it :-) > > > -Rui > > On Wed, Jul 8, 2020 at 3:55 PM Julian Hyde wrote: > >> Please call it 'externalize'. 'Serialize' gives some folks PTSD. :) >> >> On Wed, Jul 8, 2020 at 2:26 PM Rui Wang wrote: >> > >> > Thanks everyone for your inputs. Now it sounds like RexNode >> serialization >> > is not an easy effort (e.g. as easy as making classes implement >> > Serializable). I will log a JIRA to document people's opinions. >> > >> > Currently I am leaning to add serialize()/deserialize() methods to >> RexNode, >> > and also see if we can improve RelToJson/JsonToRel to include RexNode >> > serialization/deserialization (which gives JSON, thus String). >> > >> > >> > -Rui >> > >> > On Wed, Jul 8, 2020 at 1:42 PM Julian Hyde wrote: >> > >> > > Serializabilty is not very popular in Java right now. There are a >> > > bunch of concerns, including security. Serializable classes are very >> > > brittle, because it's very easy to add a non-serializable field value >> > > in a sub-class. >> > > >> > > I strongly favor externalizing over serialization. Convert RexNode to >> > > a serializable type (e.g. java.lang.String!), and then convert it back >> > > on the other side. >> > > >> > > Julian >> > > >> > > >> > > On Tue, Jul 7, 2020 at 7:25 PM Danny Chan >> wrote: >> > > > >> > > > Serialize the RexNode as Json format is a solution but I’m afraid >> it can >> > > not solve the problem completely. >> > > > One problem with it is how to re-parse the json format back to >> RexNode, >> > > the current RelJsonReader can only re-parse the RelNode but not >> RexNode, >> > > and it needs the RelOptSchema to lookup the operators. >> > > > >> > > > In the distributed scenarios of Beam, I’m afraid it is hard to get >> the >> > > RelOptSchema because it is execution, we usually see the RelOptSchema >> > > during SQL compile time. >> > > > >> > > > Best, >> > > > Danny Chan >> > > > 在 2020年7月8日 +0800 AM3:39，Roman Kondakov > >，写道： >> > > > > Hi Rui, >> > > > > >> > > > > AFAIK, RelNodes can be serialized to and deserialized from JSON >> format. >> > > > > See test [1] as an example. If I understand it correct, RelNodes >> are >> > > > > serialized along with enclosed RexNodes, so you can transfer them >> over >> > > > > the network as plain strings. >> > > > > >> > > > > [1] >> > > > > >> > > >> https://github.com/apache/calcite/blob/f64cdcbb9f6535650f0227da19640e736496a9c3/core/src/test/java/org/apache/calcite/plan/RelWriterTest.java#L88 >> > > > > >> > > > > -- >> > > > > Roman Kondakov >> > > > > >> > > > > On 07.07.2020 22:13, Enrico Olivelli wrote: >> > > > > > Rui >> > > > > > >> > > > > > Il Mar 7 Lug 2020, 20:30 Rui Wang ha >> scritto: >> > > > > > >> > > > > > > Hi Community, >> > > > > > > >> > > > > > > In Apache Beam we are facing a use case where we need to keep >> > > RexNode in >> > > > > > > our distributed primitives. Because of the nature of >> distributed >> > > computing, >> > > > > > > Beam requires the usage of those primitives be serializable >> (thus >> > > those >> > > > > > > primitives can be sent over the network to backend/workers for >> > > > > > > further execution). >> > > > > > > >> > > > > > > In the Java world this requirement means to make RexNode >> implement >> > > the Java >> > > > > > > Serializable interface. >> > > > > > > >> > > > > > > A workaround right now is to create a bunch of classes to >> "clone" >> > > RexNode >> > > > > > > while making those classes implement the Serializable >> interface. >> > > > > > > >> > > > > > >> > > > > > Did you evaluate to use some framework like Kryo that allows >> you to >> > > > > > serialize Jon serializable classes? >> > > > > > >> > > > > > I think that in general Java serialisation is not efficient as >> it is >> > > too >> > > > > > general purpose. >> > > > > > It also brings in a few Security issues. >> > > > > > >> > > > > > Maybe an alternative idea is to add some serialisation ad-hoc >> > > mechanism in >> > > > > > RexNode. >> > > > > > We should also ensure that every RexNode will be able to be >> > > serialized and >> > > > > > deserialized. >> > > > > > >> > > > > > Enrico >> > > > > > >> > > > > > >> > > > > > > So what do you think of the idea that makes RexNode implement >> the >> > > > > > > Serializable interface? >> > > > > > > >> > > > > > > >> > > > > > > -Rui >> > > > > > > >> > > > > > >> > > >> >
Re: [DISCUSS] Make RexNode serializable
O got it :-) -Rui On Wed, Jul 8, 2020 at 3:55 PM Julian Hyde wrote: > Please call it 'externalize'. 'Serialize' gives some folks PTSD. :) > > On Wed, Jul 8, 2020 at 2:26 PM Rui Wang wrote: > > > > Thanks everyone for your inputs. Now it sounds like RexNode serialization > > is not an easy effort (e.g. as easy as making classes implement > > Serializable). I will log a JIRA to document people's opinions. > > > > Currently I am leaning to add serialize()/deserialize() methods to > RexNode, > > and also see if we can improve RelToJson/JsonToRel to include RexNode > > serialization/deserialization (which gives JSON, thus String). > > > > > > -Rui > > > > On Wed, Jul 8, 2020 at 1:42 PM Julian Hyde wrote: > > > > > Serializabilty is not very popular in Java right now. There are a > > > bunch of concerns, including security. Serializable classes are very > > > brittle, because it's very easy to add a non-serializable field value > > > in a sub-class. > > > > > > I strongly favor externalizing over serialization. Convert RexNode to > > > a serializable type (e.g. java.lang.String!), and then convert it back > > > on the other side. > > > > > > Julian > > > > > > > > > On Tue, Jul 7, 2020 at 7:25 PM Danny Chan > wrote: > > > > > > > > Serialize the RexNode as Json format is a solution but I’m afraid it > can > > > not solve the problem completely. > > > > One problem with it is how to re-parse the json format back to > RexNode, > > > the current RelJsonReader can only re-parse the RelNode but not > RexNode, > > > and it needs the RelOptSchema to lookup the operators. > > > > > > > > In the distributed scenarios of Beam, I’m afraid it is hard to get > the > > > RelOptSchema because it is execution, we usually see the RelOptSchema > > > during SQL compile time. > > > > > > > > Best, > > > > Danny Chan > > > > 在 2020年7月8日 +0800 AM3:39，Roman Kondakov >，写道： > > > > > Hi Rui, > > > > > > > > > > AFAIK, RelNodes can be serialized to and deserialized from JSON > format. > > > > > See test [1] as an example. If I understand it correct, RelNodes > are > > > > > serialized along with enclosed RexNodes, so you can transfer them > over > > > > > the network as plain strings. > > > > > > > > > > [1] > > > > > > > > > https://github.com/apache/calcite/blob/f64cdcbb9f6535650f0227da19640e736496a9c3/core/src/test/java/org/apache/calcite/plan/RelWriterTest.java#L88 > > > > > > > > > > -- > > > > > Roman Kondakov > > > > > > > > > > On 07.07.2020 22:13, Enrico Olivelli wrote: > > > > > > Rui > > > > > > > > > > > > Il Mar 7 Lug 2020, 20:30 Rui Wang ha > scritto: > > > > > > > > > > > > > Hi Community, > > > > > > > > > > > > > > In Apache Beam we are facing a use case where we need to keep > > > RexNode in > > > > > > > our distributed primitives. Because of the nature of > distributed > > > computing, > > > > > > > Beam requires the usage of those primitives be serializable > (thus > > > those > > > > > > > primitives can be sent over the network to backend/workers for > > > > > > > further execution). > > > > > > > > > > > > > > In the Java world this requirement means to make RexNode > implement > > > the Java > > > > > > > Serializable interface. > > > > > > > > > > > > > > A workaround right now is to create a bunch of classes to > "clone" > > > RexNode > > > > > > > while making those classes implement the Serializable > interface. > > > > > > > > > > > > > > > > > > > Did you evaluate to use some framework like Kryo that allows you > to > > > > > > serialize Jon serializable classes? > > > > > > > > > > > > I think that in general Java serialisation is not efficient as > it is > > > too > > > > > > general purpose. > > > > > > It also brings in a few Security issues. > > > > > > > > > > > > Maybe an alternative idea is to add some serialisation ad-hoc > > > mechanism in > > > > > > RexNode. > > > > > > We should also ensure that every RexNode will be able to be > > > serialized and > > > > > > deserialized. > > > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > So what do you think of the idea that makes RexNode implement > the > > > > > > > Serializable interface? > > > > > > > > > > > > > > > > > > > > > -Rui > > > > > > > > > > > > > > > > >
Re: [DISCUSS] Make RexNode serializable
Please call it 'externalize'. 'Serialize' gives some folks PTSD. :) On Wed, Jul 8, 2020 at 2:26 PM Rui Wang wrote: > > Thanks everyone for your inputs. Now it sounds like RexNode serialization > is not an easy effort (e.g. as easy as making classes implement > Serializable). I will log a JIRA to document people's opinions. > > Currently I am leaning to add serialize()/deserialize() methods to RexNode, > and also see if we can improve RelToJson/JsonToRel to include RexNode > serialization/deserialization (which gives JSON, thus String). > > > -Rui > > On Wed, Jul 8, 2020 at 1:42 PM Julian Hyde wrote: > > > Serializabilty is not very popular in Java right now. There are a > > bunch of concerns, including security. Serializable classes are very > > brittle, because it's very easy to add a non-serializable field value > > in a sub-class. > > > > I strongly favor externalizing over serialization. Convert RexNode to > > a serializable type (e.g. java.lang.String!), and then convert it back > > on the other side. > > > > Julian > > > > > > On Tue, Jul 7, 2020 at 7:25 PM Danny Chan wrote: > > > > > > Serialize the RexNode as Json format is a solution but I’m afraid it can > > not solve the problem completely. > > > One problem with it is how to re-parse the json format back to RexNode, > > the current RelJsonReader can only re-parse the RelNode but not RexNode, > > and it needs the RelOptSchema to lookup the operators. > > > > > > In the distributed scenarios of Beam, I’m afraid it is hard to get the > > RelOptSchema because it is execution, we usually see the RelOptSchema > > during SQL compile time. > > > > > > Best, > > > Danny Chan > > > 在 2020年7月8日 +0800 AM3:39，Roman Kondakov ，写道： > > > > Hi Rui, > > > > > > > > AFAIK, RelNodes can be serialized to and deserialized from JSON format. > > > > See test [1] as an example. If I understand it correct, RelNodes are > > > > serialized along with enclosed RexNodes, so you can transfer them over > > > > the network as plain strings. > > > > > > > > [1] > > > > > > https://github.com/apache/calcite/blob/f64cdcbb9f6535650f0227da19640e736496a9c3/core/src/test/java/org/apache/calcite/plan/RelWriterTest.java#L88 > > > > > > > > -- > > > > Roman Kondakov > > > > > > > > On 07.07.2020 22:13, Enrico Olivelli wrote: > > > > > Rui > > > > > > > > > > Il Mar 7 Lug 2020, 20:30 Rui Wang ha scritto: > > > > > > > > > > > Hi Community, > > > > > > > > > > > > In Apache Beam we are facing a use case where we need to keep > > RexNode in > > > > > > our distributed primitives. Because of the nature of distributed > > computing, > > > > > > Beam requires the usage of those primitives be serializable (thus > > those > > > > > > primitives can be sent over the network to backend/workers for > > > > > > further execution). > > > > > > > > > > > > In the Java world this requirement means to make RexNode implement > > the Java > > > > > > Serializable interface. > > > > > > > > > > > > A workaround right now is to create a bunch of classes to "clone" > > RexNode > > > > > > while making those classes implement the Serializable interface. > > > > > > > > > > > > > > > > Did you evaluate to use some framework like Kryo that allows you to > > > > > serialize Jon serializable classes? > > > > > > > > > > I think that in general Java serialisation is not efficient as it is > > too > > > > > general purpose. > > > > > It also brings in a few Security issues. > > > > > > > > > > Maybe an alternative idea is to add some serialisation ad-hoc > > mechanism in > > > > > RexNode. > > > > > We should also ensure that every RexNode will be able to be > > serialized and > > > > > deserialized. > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > So what do you think of the idea that makes RexNode implement the > > > > > > Serializable interface? > > > > > > > > > > > > > > > > > > -Rui > > > > > > > > > > > > >
Re: [DISCUSS] Make RexNode serializable
Thanks everyone for your inputs. Now it sounds like RexNode serialization is not an easy effort (e.g. as easy as making classes implement Serializable). I will log a JIRA to document people's opinions. Currently I am leaning to add serialize()/deserialize() methods to RexNode, and also see if we can improve RelToJson/JsonToRel to include RexNode serialization/deserialization (which gives JSON, thus String). -Rui On Wed, Jul 8, 2020 at 1:42 PM Julian Hyde wrote: > Serializabilty is not very popular in Java right now. There are a > bunch of concerns, including security. Serializable classes are very > brittle, because it's very easy to add a non-serializable field value > in a sub-class. > > I strongly favor externalizing over serialization. Convert RexNode to > a serializable type (e.g. java.lang.String!), and then convert it back > on the other side. > > Julian > > > On Tue, Jul 7, 2020 at 7:25 PM Danny Chan wrote: > > > > Serialize the RexNode as Json format is a solution but I’m afraid it can > not solve the problem completely. > > One problem with it is how to re-parse the json format back to RexNode, > the current RelJsonReader can only re-parse the RelNode but not RexNode, > and it needs the RelOptSchema to lookup the operators. > > > > In the distributed scenarios of Beam, I’m afraid it is hard to get the > RelOptSchema because it is execution, we usually see the RelOptSchema > during SQL compile time. > > > > Best, > > Danny Chan > > 在 2020年7月8日 +0800 AM3:39，Roman Kondakov ，写道： > > > Hi Rui, > > > > > > AFAIK, RelNodes can be serialized to and deserialized from JSON format. > > > See test [1] as an example. If I understand it correct, RelNodes are > > > serialized along with enclosed RexNodes, so you can transfer them over > > > the network as plain strings. > > > > > > [1] > > > > https://github.com/apache/calcite/blob/f64cdcbb9f6535650f0227da19640e736496a9c3/core/src/test/java/org/apache/calcite/plan/RelWriterTest.java#L88 > > > > > > -- > > > Roman Kondakov > > > > > > On 07.07.2020 22:13, Enrico Olivelli wrote: > > > > Rui > > > > > > > > Il Mar 7 Lug 2020, 20:30 Rui Wang ha scritto: > > > > > > > > > Hi Community, > > > > > > > > > > In Apache Beam we are facing a use case where we need to keep > RexNode in > > > > > our distributed primitives. Because of the nature of distributed > computing, > > > > > Beam requires the usage of those primitives be serializable (thus > those > > > > > primitives can be sent over the network to backend/workers for > > > > > further execution). > > > > > > > > > > In the Java world this requirement means to make RexNode implement > the Java > > > > > Serializable interface. > > > > > > > > > > A workaround right now is to create a bunch of classes to "clone" > RexNode > > > > > while making those classes implement the Serializable interface. > > > > > > > > > > > > > Did you evaluate to use some framework like Kryo that allows you to > > > > serialize Jon serializable classes? > > > > > > > > I think that in general Java serialisation is not efficient as it is > too > > > > general purpose. > > > > It also brings in a few Security issues. > > > > > > > > Maybe an alternative idea is to add some serialisation ad-hoc > mechanism in > > > > RexNode. > > > > We should also ensure that every RexNode will be able to be > serialized and > > > > deserialized. > > > > > > > > Enrico > > > > > > > > > > > > > So what do you think of the idea that makes RexNode implement the > > > > > Serializable interface? > > > > > > > > > > > > > > > -Rui > > > > > > > > > >
Re: [DISCUSS] Make RexNode serializable
Serializabilty is not very popular in Java right now. There are a bunch of concerns, including security. Serializable classes are very brittle, because it's very easy to add a non-serializable field value in a sub-class. I strongly favor externalizing over serialization. Convert RexNode to a serializable type (e.g. java.lang.String!), and then convert it back on the other side. Julian On Tue, Jul 7, 2020 at 7:25 PM Danny Chan wrote: > > Serialize the RexNode as Json format is a solution but I’m afraid it can not > solve the problem completely. > One problem with it is how to re-parse the json format back to RexNode, the > current RelJsonReader can only re-parse the RelNode but not RexNode, and it > needs the RelOptSchema to lookup the operators. > > In the distributed scenarios of Beam, I’m afraid it is hard to get the > RelOptSchema because it is execution, we usually see the RelOptSchema during > SQL compile time. > > Best, > Danny Chan > 在 2020年7月8日 +0800 AM3:39，Roman Kondakov ，写道： > > Hi Rui, > > > > AFAIK, RelNodes can be serialized to and deserialized from JSON format. > > See test [1] as an example. If I understand it correct, RelNodes are > > serialized along with enclosed RexNodes, so you can transfer them over > > the network as plain strings. > > > > [1] > > https://github.com/apache/calcite/blob/f64cdcbb9f6535650f0227da19640e736496a9c3/core/src/test/java/org/apache/calcite/plan/RelWriterTest.java#L88 > > > > -- > > Roman Kondakov > > > > On 07.07.2020 22:13, Enrico Olivelli wrote: > > > Rui > > > > > > Il Mar 7 Lug 2020, 20:30 Rui Wang ha scritto: > > > > > > > Hi Community, > > > > > > > > In Apache Beam we are facing a use case where we need to keep RexNode in > > > > our distributed primitives. Because of the nature of distributed > > > > computing, > > > > Beam requires the usage of those primitives be serializable (thus those > > > > primitives can be sent over the network to backend/workers for > > > > further execution). > > > > > > > > In the Java world this requirement means to make RexNode implement the > > > > Java > > > > Serializable interface. > > > > > > > > A workaround right now is to create a bunch of classes to "clone" > > > > RexNode > > > > while making those classes implement the Serializable interface. > > > > > > > > > > Did you evaluate to use some framework like Kryo that allows you to > > > serialize Jon serializable classes? > > > > > > I think that in general Java serialisation is not efficient as it is too > > > general purpose. > > > It also brings in a few Security issues. > > > > > > Maybe an alternative idea is to add some serialisation ad-hoc mechanism in > > > RexNode. > > > We should also ensure that every RexNode will be able to be serialized and > > > deserialized. > > > > > > Enrico > > > > > > > > > > So what do you think of the idea that makes RexNode implement the > > > > Serializable interface? > > > > > > > > > > > > -Rui > > > > > > >
Re: Question about exposing additional built-in operators
Add something like this to SqlLibraryOperators:
@LibraryOperator(libraries = {POSTGRESQL})
public static final SqlFunction LENGTH = ...;
and create your JDBC connection with 'fun=postgresql'.
LENGTH is not part of the SQL standard, so it should not be in
SqlStdOperatorTable.
Nor should you add SQL standard operators to SqlLibraryOperators; they
belong in SqlStdOperatorTable.
There is not a library for Presto but we could consider adding one.
It's probably only worthwhile if it is significantly different from
other DBs.
Julian
On Wed, Jul 8, 2020 at 11:50 AM James Anderson wrote:
>
> Hi,
>
> Is there a mechanism to add/register additional standard operators to be
> used by Calcite when defining a model for a JDBC connection? I had hoped
> that annotating new functions with LibraryOperator(libraries={STANDARD})
> would work but it doesn't seem to be the case, or I'm doing it incorrectly.
>
> My use case is that I'd like to port a query between two different backends
> (Presto, Postgres) that both offer a built-in operator (LENGTH, for
> example), which isn't part of SqlStdOperatorTable but I'd like to be able
> to support validating queries that call that function without creating my
> own FrameworkConfig, Planner, etc.
>
> Thanks!
> James
Question about exposing additional built-in operators
Hi,
Is there a mechanism to add/register additional standard operators to be
used by Calcite when defining a model for a JDBC connection? I had hoped
that annotating new functions with LibraryOperator(libraries={STANDARD})
would work but it doesn't seem to be the case, or I'm doing it incorrectly.
My use case is that I'd like to port a query between two different backends
(Presto, Postgres) that both offer a built-in operator (LENGTH, for
example), which isn't part of SqlStdOperatorTable but I'd like to be able
to support validating queries that call that function without creating my
own FrameworkConfig, Planner, etc.
Thanks!
James
[jira] [Created] (CALCITE-4113) Support LEFT join in EnumerableMergeJoin
Ruben Q L created CALCITE-4113: -- Summary: Support LEFT join in EnumerableMergeJoin Key: CALCITE-4113 URL: https://issues.apache.org/jira/browse/CALCITE-4113 Project: Calcite Issue Type: New Feature Components: core Reporter: Ruben Q L Currently, EnumerableMergeJoin supports INNER, SEMI, and ANTI join. The goal of this ticket is to support also LEFT join (which should be a combination of INNER join results + ANTI join results with a NULL on the right). LEFT EnumerableMergeJoin should maintain the order of the left-hand-side of the join. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4112) Better judgment "config.forceDecorrelate" in DecorrelateProgram
Jiatao Tao created CALCITE-4112: --- Summary: Better judgment "config.forceDecorrelate" in DecorrelateProgram Key: CALCITE-4112 URL: https://issues.apache.org/jira/browse/CALCITE-4112 Project: Calcite Issue Type: Improvement Components: core Reporter: Jiatao Tao Assignee: Jiatao Tao -- This message was sent by Atlassian Jira (v8.3.4#803005)
[DISCUSS] Default disable the RexNode normalization(or operands reorder)
In CALCITE-2450, we proposed a change to normalize the RexNode, and there is a discussion[1], the change is in very early phrase and the normalization pattern is unstable. There is actually no common consensus about what a form (or pattern) a desired normalization should be: • People may have different requests in different contexts. • Different downstream projects may also have different requests The problem becomes critical after CALCITE-3786 because there are more cases be normalized (about 50+ plan changes). In CALCITE-3786, we move the normalization to constructor because the digest equals and object equals should be equivalent for the RexCalls. The downstream project like Apache Flink would have much more cases with normalized plans. But actually, the normalization gains little. I think other downstream projects have similar situation. I would suggest to default disable the normalization until it is “stable” enough, at least, after we have a consensus about what is a normalized pattern should be, there is an issue [3] already and we can have more discussion based on that. Appreciate for your suggestions, thanks in advance ~ [1] https://lists.apache.org/x/thread.html/54bf3ed733eb7e725ce3ea397334aad8f1323ead13e450b1753b1521@%3Cdev.calcite.apache.org%3E [2] https://issues.apache.org/jira/browse/CALCITE-2450 [3] https://issues.apache.org/jira/browse/CALCITE-4073 Best, Danny Chan
