Hello, Andrey. > Ignite SQL layer has some issues that can't be fix with changes in Ignite > only, and we are blocked with H2.
What are these issues? Can you make it specific and send a tickets for this issues? > 3. Replace H2 with smth else. Actually, I support this decision in general. But, to make a right choise for H2 replacement we should carefully discuss such huge replacement. So far, I can't see any written down(in IEP) requirements for SQL engine. Let's do it and discuss them. В Пт, 27/09/2019 в 13:39 +0300, Andrey Mashenkov пишет: > Hi Nikolay, > > Let me add my 5- cent here. > > Ignite SQL layer has some issues that can't be fix with changes in Ignite > only, and we are blocked with H2. > To resolve these issues we can: > 1. Donate some changes to H2 and wait for it's next release. But there are > more cons than pros and I think we can't rely on H2 project anymore. > - There is no guarantee our changes will be approved by H2 community. > - We definitely won't to depend on H2 product lifecycle. > - New H2 features (like parallel multi-statement query processing in latest > release) force Ignite for significant changes\refactoring in Ignite SQL > layer with no visible benefits. > Every next release it becomes harder to upgrade H2 dependency. > - Latest H2 versions causes questions about their stability. > > Hot issues are > - Large intermediate results inside H2 internals can cause OOM for some > kind of queries. Ignite can't handle this anyhow for now without reworking > H2 code. > - HashJoins > - Ignite can't start multi-step queries, but 2-step (map-reduce) only. > - It is not possible to apply optimizations on query plan as no logical > plan actually doen't exists. H2 execution plan is hard-wired with H2 > internals and can't be easily transformed. > Implementing a new good planner over H2 looks like a huge task. > > 2. Fork H2. > We already done this in GridGain (you can found H2 module in GridGain > community edition) as fastest way to unblock work on SQL improvements. > But this way doesn't look like a good one for Ignite, regarding our > experience. > - H2 code can't be included into Ignite at all. > H2 license are MIT and EPL. From one side they can't be changed to Apache > Licence. From other side Apache Foundation don't want to host any code > licensed with other than Apache License. > GridGain is ok with this, but Apache Foundation won't. > > - We can made separate H2 fork project with it's own lifecycle with full > control over it and publish it in Maven Central. > This doen't seem like a big deal. But will causes additional difficulties > in development, test and release processes of Ignite. > This way seems bring much pain for every contributor. > > 3. Replace H2 with smth else. > E.g. with Apache Calcite. > - Calcite is a framework and it is designed very flexible and extendable. > - Every it's part can be replaced with our own implementation. > - Apache License is out of the box =) > > So, summary: > 1-st way of pain we have now and it slows down Ignite SQL layer developing. > 2-nd looks few better, but seems bring Ignite to nowhere in prospect. > 3-rd is a risky, but promissory way. > > > On Fri, Sep 27, 2019 at 12:16 PM Nikolay Izhikov <nizhi...@apache.org> > wrote: > > > Hello, Igor. > > > > Thanks for starting this discussion. > > > > I think we should take a step back in it and answer the following > > questions: > > > > 1. What the exact issues with the H2 integration? > > Can you send a tickets links? > > Can we label all H2 integration issues in JIRA? I propose to use "h2" > > label. > > > > 2. What are the requirements for the new SQL engine? > > We should write it down and discuss. > > > > 3. What options do we have? > > Are there any alternatives to Calcite on the market? > > We did the wrong choice that looked obvious one time. > > So we should carefully avoid it at this time. > > > > 4. What is improvements of Ignite we want to make with the new engine? > > > > > > В Пт, 27/09/2019 в 08:44 +0000, Igor Seliverstov пишет: > > > Hi Igniters! > > > > > > As you might know currently we have many open issues relating to current > > > > H2 based engine and its execution flow. > > > > > > Some of them are critical (like impossibility to execute particular > > > > queries), some of them are majors (like impossibility to execute particular > > queries without pre-preparation your data to have a collocation) and many > > minors. > > > > > > Most of the issues cannot be solved without whole engine redesign. > > > > > > So, here the proposal: > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=130028084 > > > > > > I'll appreciate if you share your thoughts on top of that. > > > > > > Regards, > > > Igor > >
signature.asc
Description: This is a digitally signed message part
