+1 for this idea, I have been developing Calcite for a long time(counting during project Kylin), we all treat calcite as an optimizer, but we need to consider overhead.
I aggre with Stamatis: "since those dependencies were not causing any real trouble." What really troubling me is that when we do some in logical, we may have to consider the implemnt, for an example, we used keep "In", not convert to join or "OR", but calcite have no impl about "In". Regards! Aron Tao Haisheng Yuan <[email protected]> 于2020年11月25日周三 下午12:57写道： > > I would like to propose to decouple the "core" module from "ling4j" and > Avatica. > I like the idea. > > Moving Enumerable out of core may be time consuming and disruptive, > because many core tests are using Enumerable to verify plan quality and > correctness. > > Best, > Haisheng > > On 2020/11/24 23:42:19, Stamatis Zampetakis <[email protected]> wrote: > > Hi Vladimir, > > > > Personally, I like the idea. > > I had similar thoughts in the past but it didn't try to break it down > since > > those dependencies were not causing any real trouble. > > > > Let's see what the others think. > > > > Best, > > Stamatis > > > > > > On Tue, Nov 24, 2020 at 7:30 PM Vladimir Ozerov <[email protected]> > wrote: > > > > > Hi colleagues, > > > > > > Many Calcite integrations use only part of the framework. > Specifically, it > > > is common to use only the parser/optimizer part. JDBC and runtime are > used > > > less frequently because they are not very well suited for mature > processing > > > engines (e.g. Enumerable runs out of memory easily). > > > > > > However, in order to use the parser/optimizer from the core module, you > > > also need to add "linq4j" and Avatica modules to the classpath, which > is > > > not convenient - why to include modules, that you do not use? > > > > > > It turns out that most of the dependencies are indeed leaky > abstractions, > > > that could be decoupled easily. For example, the RelOptUtil class from > the > > > "core" depends on ... two string constants from the Avatica module. > > > > > > I would like to propose to decouple the "core" module from "ling4j" and > > > Avatica. For example, we may introduce the new "common" module, that > will > > > hold common constants, utility classes, and interfaces (e.g. Meta). > Then, > > > we can organize the dependencies like this: > > > common -> core > > > common -> linq4j > > > common -> Avatica > > > > > > Finally, we may shade and relocate the "common" module into the "core" > > > during the build. In the end, we will have -2 runtime dependencies with > > > relatively little effort. In principle, the same approach could be > applied > > > to Janino and Jackson dependencies, but it could be more complex, so my > > > proposal is only about "linq4" and Avatica. > > > > > > How do you feel about it? Does this proposal sense to the community? If > > > yes, I can try implementing the POC for this. > > > > > > Regards, > > > Vladimir. > > > > > >
