Maxim, Nikolay,
I've listed two issues which show the ideological flaws of the current
engine.
1. IGNITE-11448 - Open. This ticket describes the impossibility of
executing queries which can not be fit in the hardcoded one pass
map-reduce paradigm.
2. IGNITE-6085 - Closed (won't fix) - This ticket describes the second
major problem with the current engine: H2 query optimizer is very
primitive and can not perform many useful optimizations.
These two points (single map-reduce execution and inflexible optimizer)
are the main problems with the current engine. It means that our engine
is currently suitable for execution only a very limited subset of the
typical SQL queries. For example it can not even run most of the TPC-H
benchmark queries because they don't fit to the simple map-reduce paradigm.
All I see is links to two tickets:
How many tickets would satisfy you? I named two. And it looks like it is
not enough from your point of view. Ok, so how many is enough? The set
of problems caused by listed above tickets is infinite, therefore I can
not create a ticket for each of them.
Tech details also should be added.
Tech details are in the tickets.
We can't discuss such a huge change as an execution engine replacement with
descrition like:
"No data co-location control, i.e. arbitrary data can be returned silently" or
"Low control on how query executes internally, as a result we have limited
possibility to implement improvements/fixes."
Why not? Don't you understand these problems? Or you don't think this is
a problem?
Let's make these descriptions more specific.
What do you mean by "more specific"? What is the criteria of the
specific description?
Nikolay, Maxim, I understand that our arguments may not be as obvious
for you as it obvious for SQL team. So, please arrange your questions in
a more constructive way.
Thank you!
--
Kind Regards
Roman Kondakov
On 27.09.2019 15:32, Maxim Muzafarov wrote:
Folks,
I agree with Nikolay, the idea of replacing the H2 engine with the
most suitable one is reasonable. But since such change is major we
should have a strong argumentation on it even for members with are
working outside the SQL-team.
I think it is really necessary to have:
1. The list of issues related to the current engine (H2) which from
different points of view and for different developers must seem
unsolvable. For example, `... the H2 execution plan is hard-wired with
H2 internals and can't be easily transformed` seems doesn't have a
strong technical argumentation.
After this step, we should have a clear understanding that the engine
change is required.
2. Why only the Apache Calcite? It seems to me we should have a table
with a comparison of different engines with the pros and cons of each
other. A brief search shows me that we may have a few options here.
After this step, we should have a clear understanding of why we choose
this dependency prior to another.
3. We should also have a migration decomposition and step by step
actions to do. I haven't found such a decomposition on IEP-37 page. Do
we have one? What the implementation phases will be? What components
will be changed? What a new API would be and would it be? What
problems we are expecting e.g performance degradation on prototype
implementation? `Risks and Assumptions` topic doesn't seem to be a
good described.
After this step, we should have a clear and obvious a new feature
implementation plan.
Let's have a strong technical discussion.
On Fri, 27 Sep 2019 at 15:17, Nikolay Izhikov <nizhi...@apache.org> wrote:
Hello, Roman.
All I see is links to two tickets:
IGNITE-11448 - Open
IGNITE-6085 - Closed
Other issues described poorly and have not ticket links.
We can't discuss such a huge change as an execution engine replacement with
descrition like:
"No data co-location control, i.e. arbitrary data can be returned silently" or
"Low control on how query executes internally, as a result we have limited
possibility to implement improvements/fixes."
I think we need some reproducer that shows issue.
Tech details also should be added.
Let's make these descriptions more specific.
Let's discuss how we want to fix them with the new engine.
В Пт, 27/09/2019 в 15:10 +0300, Roman Kondakov пишет:
Hello Nikolay,
please see IEP--37 [1]. Issues are there.
[1]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=130028084
