[jira] [Commented] (IGNITE-14593) Calcite. Support not equal expression.
[
https://issues.apache.org/jira/browse/IGNITE-14593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17325530#comment-17325530
]
Roman Kondakov commented on IGNITE-14593:
-
What's wrong with {{<>}} sign?
> Calcite. Support not equal expression.
> --
>
> Key: IGNITE-14593
> URL: https://issues.apache.org/jira/browse/IGNITE-14593
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Stanilovsky Evgeny
>Priority: Major
> Labels: calcite
>
> Looks like it would be helpful to support "!=" syntax, i.e. :
> {code:java}
> UPDATE test SET a=7 WHERE id != 3
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13125) SQL regressions framework: Random query generation
[ https://issues.apache.org/jira/browse/IGNITE-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174171#comment-17174171 ] Roman Kondakov commented on IGNITE-13125: - Hi [~korlov]! Feel free to pick up this ticket. Unfortunately I don't have enough time to complete this task right now. Thanks! > SQL regressions framework: Random query generation > -- > > Key: IGNITE-13125 > URL: https://issues.apache.org/jira/browse/IGNITE-13125 > Project: Ignite > Issue Type: Test > Components: sql >Reporter: Roman Kondakov >Assignee: Roman Kondakov >Priority: Major > > We need to build a framework for random SQL query generation. At first > iteration it should be able to generate at least randomized joins queries. > As a reference implementation we can take > [SqlSmith|https://github.com/anse1/sqlsmith] or build something similar to it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13021) Calcite integration. Avoid full scans for disjunctive queries.
[
https://issues.apache.org/jira/browse/IGNITE-13021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130565#comment-17130565
]
Roman Kondakov commented on IGNITE-13021:
-
[~amashenkov] looks good to me.
> Calcite integration. Avoid full scans for disjunctive queries.
> --
>
> Key: IGNITE-13021
> URL: https://issues.apache.org/jira/browse/IGNITE-13021
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Andrey Mashenkov
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Currently a full table scan will be executed in the case of disjunctive
> predicate even if predicate fields are indexed. For example:
> {code:java}
> SELECT * FROM emps WHERE name='A' OR surname='B'
> {code}
> This is caused by the nature of indexes: they can return cursor bounded by
> lower and upper bounds. We can cope with it by implementing a logical rule
> for rewriting {{OR}} query to a {{UNION ALL}} query:
> {code:java}
> SELECT * FROM emps WHERE name='A'
> UNION ALL
> SELECT * FROM emps WHERE surname='B' AND LNNVL(name='A')
> {code}
> where {{LNNVL()}} function has semantics
> {code:java}
> LNNVL(name='A') == name!='A' OR name=NULL.
> {code}
> It is used to avoid expensive deduplication. This name is taken from Oracle,
> we can think of more meaningful name, or find the analog in Calcite or H2.
> See, for example, this blog post:
> [https://blogs.oracle.com/optimizer/optimizer-transformations:-or-expansion]
> for details.
> Also it is needed to check this works for {{IN}} clause with small number of
> literals (AFAIK Calcite converts large {{IN}} clauses to a join with
> {{Values}} table where N > 20).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13136) Calcite integration. Improve join predicate testing.
[
https://issues.apache.org/jira/browse/IGNITE-13136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-13136:
Description:
Currently we have to merge joining rows in order to test a join predicate:
{code:java}
Row row = handler.concat(left, rightMaterialized.get(rightIdx++));
if (!cond.test(row))
continue;
{code}
it results in unconditional building a joined row even if it will not be
emitted to downstream further. To avoid extra GC pressure we need to test the
join predicate before joining rows:
{code:java}
if (!cond.test(left, right))
continue;
Row row = handler.concat(left, right);
{code}
was:
Currently we have to merge joining rows in order to test a join predicate:
{code:java}
Row row = handler.concat(left, rightMaterialized.get(rightIdx++));
if (!cond.test(row))
continue;
{code}
it results in unconditional building a joining row even if it will not be
emitted to downstream further. To avoid extra GC pressure we need to test the
join predicate before joining rows:
{code:java}
if (!cond.test(left, right))
continue;
Row row = handler.concat(left, right);
{code}
> Calcite integration. Improve join predicate testing.
>
>
> Key: IGNITE-13136
> URL: https://issues.apache.org/jira/browse/IGNITE-13136
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Priority: Minor
>
> Currently we have to merge joining rows in order to test a join predicate:
> {code:java}
> Row row = handler.concat(left, rightMaterialized.get(rightIdx++));
> if (!cond.test(row))
> continue;
> {code}
> it results in unconditional building a joined row even if it will not be
> emitted to downstream further. To avoid extra GC pressure we need to test the
> join predicate before joining rows:
> {code:java}
> if (!cond.test(left, right))
> continue;
> Row row = handler.concat(left, right);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13136) Calcite integration. Improve join predicate testing.
Roman Kondakov created IGNITE-13136:
---
Summary: Calcite integration. Improve join predicate testing.
Key: IGNITE-13136
URL: https://issues.apache.org/jira/browse/IGNITE-13136
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Roman Kondakov
Currently we have to merge joining rows in order to test a join predicate:
{code:java}
Row row = handler.concat(left, rightMaterialized.get(rightIdx++));
if (!cond.test(row))
continue;
{code}
it results in unconditional building a joining row even if it will not be
emitted to downstream further. To avoid extra GC pressure we need to test the
join predicate before joining rows:
{code:java}
if (!cond.test(left, right))
continue;
Row row = handler.concat(left, right);
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13021) Calcite integration. Avoid full scans for disjunctive queries.
[
https://issues.apache.org/jira/browse/IGNITE-13021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17129084#comment-17129084
]
Roman Kondakov commented on IGNITE-13021:
-
[~amashenkov] The patch looks good for me. I left a single comment: I think we
need to ensure the proper behavior in the presence of {{null}} values.
> Calcite integration. Avoid full scans for disjunctive queries.
> --
>
> Key: IGNITE-13021
> URL: https://issues.apache.org/jira/browse/IGNITE-13021
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Andrey Mashenkov
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently a full table scan will be executed in the case of disjunctive
> predicate even if predicate fields are indexed. For example:
> {code:java}
> SELECT * FROM emps WHERE name='A' OR surname='B'
> {code}
> This is caused by the nature of indexes: they can return cursor bounded by
> lower and upper bounds. We can cope with it by implementing a logical rule
> for rewriting {{OR}} query to a {{UNION ALL}} query:
> {code:java}
> SELECT * FROM emps WHERE name='A'
> UNION ALL
> SELECT * FROM emps WHERE surname='B' AND LNNVL(name='A')
> {code}
> where {{LNNVL()}} function has semantics
> {code:java}
> LNNVL(name='A') == name!='A' OR name=NULL.
> {code}
> It is used to avoid expensive deduplication. This name is taken from Oracle,
> we can think of more meaningful name, or find the analog in Calcite or H2.
> See, for example, this blog post:
> [https://blogs.oracle.com/optimizer/optimizer-transformations:-or-expansion]
> for details.
> Also it is needed to check this works for {{IN}} clause with small number of
> literals (AFAIK Calcite converts large {{IN}} clauses to a join with
> {{Values}} table where N > 20).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (IGNITE-13125) SQL regressions framework: Random query generation
[ https://issues.apache.org/jira/browse/IGNITE-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov reassigned IGNITE-13125: --- Assignee: Roman Kondakov > SQL regressions framework: Random query generation > -- > > Key: IGNITE-13125 > URL: https://issues.apache.org/jira/browse/IGNITE-13125 > Project: Ignite > Issue Type: Test > Components: sql >Reporter: Roman Kondakov >Assignee: Roman Kondakov >Priority: Major > > We need to build a framework for random SQL query generation. At first > iteration it should be able to generate at least randomized joins queries. > As a reference implementation we can take > [SqlSmith|https://github.com/anse1/sqlsmith] or build something similar to it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13125) SQL regressions framework: Random query generation
Roman Kondakov created IGNITE-13125: --- Summary: SQL regressions framework: Random query generation Key: IGNITE-13125 URL: https://issues.apache.org/jira/browse/IGNITE-13125 Project: Ignite Issue Type: Test Components: sql Reporter: Roman Kondakov We need to build a framework for random SQL query generation. At first iteration it should be able to generate at least randomized joins queries. As a reference implementation we can take [SqlSmith|https://github.com/anse1/sqlsmith] or build something similar to it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13070) SQL regressions detection framework
[ https://issues.apache.org/jira/browse/IGNITE-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126602#comment-17126602 ] Roman Kondakov commented on IGNITE-13070: - Tests look good [https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_PdsCompatibility/5365893?buildTab=overview] > SQL regressions detection framework > --- > > Key: IGNITE-13070 > URL: https://issues.apache.org/jira/browse/IGNITE-13070 > Project: Ignite > Issue Type: Test > Components: sql >Reporter: Roman Kondakov >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > We need to detect SQL regressions early. We can do it by comparing the SQL > query performance for different Ignite versions. This test framework should > work in the following way: > # It starts two Ignite clusters with different versions (current version and > the previous release version). > # Framework then runs randomly generated queries in both clusters and checks > the execution time for each cluster. We need to port SQLSmith library from > C++ to java for this step. But initially we can start with some set of > hardcoded queries and postpone the SQLSmith port. Randomized queries can be > added later. > # All problematic queries are then reported as performance issues. In this > way we can manually examine the problems. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12909) Calcite integration. Explain command syntax change
[
https://issues.apache.org/jira/browse/IGNITE-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12909:
Description: We need to change the default Calcite syntax from {{EXPLAIN
PLAN FOR}} to {{EXPLAIN}}. (was: Support EXPLAIN command)
> Calcite integration. Explain command syntax change
> --
>
> Key: IGNITE-12909
> URL: https://issues.apache.org/jira/browse/IGNITE-12909
> Project: Ignite
> Issue Type: Task
>Reporter: Igor Seliverstov
>Priority: Major
>
> We need to change the default Calcite syntax from {{EXPLAIN PLAN FOR}} to
> {{EXPLAIN}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12909) Calcite integration. Explain command syntax change
[
https://issues.apache.org/jira/browse/IGNITE-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12909:
Priority: Minor (was: Major)
> Calcite integration. Explain command syntax change
> --
>
> Key: IGNITE-12909
> URL: https://issues.apache.org/jira/browse/IGNITE-12909
> Project: Ignite
> Issue Type: Task
>Reporter: Igor Seliverstov
>Priority: Minor
>
> We need to change the default Calcite syntax from {{EXPLAIN PLAN FOR}} to
> {{EXPLAIN}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12909) Calcite integration. Explain command syntax change
[ https://issues.apache.org/jira/browse/IGNITE-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12909: Summary: Calcite integration. Explain command syntax change (was: Calcite integration. Explain command support) > Calcite integration. Explain command syntax change > -- > > Key: IGNITE-12909 > URL: https://issues.apache.org/jira/browse/IGNITE-12909 > Project: Ignite > Issue Type: Task >Reporter: Igor Seliverstov >Priority: Major > > Support EXPLAIN command -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12899) Calcite integration. Distribution multitrait
[ https://issues.apache.org/jira/browse/IGNITE-12899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12899: Priority: Minor (was: Major) > Calcite integration. Distribution multitrait > > > Key: IGNITE-12899 > URL: https://issues.apache.org/jira/browse/IGNITE-12899 > Project: Ignite > Issue Type: Task >Reporter: Igor Seliverstov >Priority: Minor > > Currently Ignite nodes have single distribution value which isn't true in > several cases like: > # a partitioned table with a key alias should have 2 distribution traits: by > _key and by key alias (id for example) - without distribution by _key column > almost impossible to implement IGNITE-12692 > # After inner / cross join the result should have several hash distributions > (by left and by right keys) - it may be important on join transpose > optimization > # On project a source distribution key may appear more than once, so the > result should contain a distribution for each distribution key position. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12819) Calcite integration. Cost system.
[ https://issues.apache.org/jira/browse/IGNITE-12819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12819: Priority: Minor (was: Major) > Calcite integration. Cost system. > - > > Key: IGNITE-12819 > URL: https://issues.apache.org/jira/browse/IGNITE-12819 > Project: Ignite > Issue Type: Task >Reporter: Igor Seliverstov >Priority: Minor > > Current implementation doesn't suit our needs for particular reasons. We need > to introduce our own one taking into account such parameters as IO > operations, memory usage, CPU usage, disk usage etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12563) Calcite integration. Reset and remap queries on topology change.
[ https://issues.apache.org/jira/browse/IGNITE-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12563: Priority: Minor (was: Major) > Calcite integration. Reset and remap queries on topology change. > > > Key: IGNITE-12563 > URL: https://issues.apache.org/jira/browse/IGNITE-12563 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Igor Seliverstov >Priority: Minor > > A query should: > * be silently remapped in case an actual topology version at query plan > materialization time differs from a topology version at the time locations > was calculated > * be cancelled with appropriate exception in case one of responsible nodes > left a cluster after execution is started. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13035) Calcite integration. Support fallback on indexes rebuild
[ https://issues.apache.org/jira/browse/IGNITE-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-13035: Priority: Minor (was: Major) > Calcite integration. Support fallback on indexes rebuild > > > Key: IGNITE-13035 > URL: https://issues.apache.org/jira/browse/IGNITE-13035 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Roman Kondakov >Priority: Minor > > When indexes are being rebuilt we need to use CacheTree for index scans as it > happens in legacy engine. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13027) Calcite integration. Query parallelism support.
[ https://issues.apache.org/jira/browse/IGNITE-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-13027: Priority: Minor (was: Major) > Calcite integration. Query parallelism support. > --- > > Key: IGNITE-13027 > URL: https://issues.apache.org/jira/browse/IGNITE-13027 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Roman Kondakov >Priority: Minor > > We need to add support for segmented indexes in Calcite engine. Currently > engine is able to execute queries over index with only one segment. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13030) Calcite integration. Push projections to scans and avoid reading full row when possible
[
https://issues.apache.org/jira/browse/IGNITE-13030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-13030:
Priority: Minor (was: Major)
> Calcite integration. Push projections to scans and avoid reading full row
> when possible
> ---
>
> Key: IGNITE-13030
> URL: https://issues.apache.org/jira/browse/IGNITE-13030
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Priority: Minor
>
> Currently {{IngiteTableScan}} supports only embedded filters. If it supports
> also embedded projections (see {{ProjectableFilterableTable}} in Calcite), it
> will be able to push these projections down to page memory scans in order to
> avoid full row reading.
> For example for the query
> {code:java}
> SELECT _val FROM tbl
> {code}
> we can read only values.
> Moreover, we can establish index-only scans in some cases. For example, if
> the column {{depId}} is indexed, we can the execute query
> {code:java}
> SELECT depId FROM emp
> {code}
> using only index tree, without reading rows at all.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13023) Calcite integration. Page scan support.
[ https://issues.apache.org/jira/browse/IGNITE-13023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-13023: Priority: Minor (was: Major) > Calcite integration. Page scan support. > --- > > Key: IGNITE-13023 > URL: https://issues.apache.org/jira/browse/IGNITE-13023 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Roman Kondakov >Priority: Minor > > We need to evaluate the advantage of using data page scan (see IGNITE-10798) > in Calcite engine and support it if needed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13022) Calcite integration. Merge index conditions for the same field.
[
https://issues.apache.org/jira/browse/IGNITE-13022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-13022:
Priority: Minor (was: Major)
> Calcite integration. Merge index conditions for the same field.
> ---
>
> Key: IGNITE-13022
> URL: https://issues.apache.org/jira/browse/IGNITE-13022
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Priority: Minor
>
> Index scans should be able to merge index conditions. For example query
> {code:java}
> SELECT * FROM tbl WHERE a<5 AND a<10
> {code}
> should be reduced to
>
> {code:java}
> SELECT * FROM tbl WHERE a<5
> {code}
> Parameters should be handled in a more tricky way:
> {code:java}
> SELECT * FROM tbl WHERE a {code}
> can be rewritten as
> {code:java}
> SELECT * FROM tbl WHERE a where the expression {{MIN(?1, ?2)}} should be evaluated right before the
> execution when parameters are known.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12991) Calcite integration. Pass cancel flag to VolcanoPlanner
[
https://issues.apache.org/jira/browse/IGNITE-12991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12991:
Priority: Minor (was: Major)
> Calcite integration. Pass cancel flag to VolcanoPlanner
> ---
>
> Key: IGNITE-12991
> URL: https://issues.apache.org/jira/browse/IGNITE-12991
> Project: Ignite
> Issue Type: Improvement
>Reporter: Igor Seliverstov
>Priority: Minor
>
> see {{AbstractRelOptPlanner.java:91}}, here {{CancelFlag}} is used to cancel
> planning loop. We need to pass it into a newly created context and bind its
> state with {{PlanningContext#queryCancel}} to break possible infinite
> planning loop. See also {{PlanningContext#unwrap}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12914) Calcite integration: Add aggregate project merge rule to the planner
[
https://issues.apache.org/jira/browse/IGNITE-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12914:
Priority: Minor (was: Major)
> Calcite integration: Add aggregate project merge rule to the planner
>
>
> Key: IGNITE-12914
> URL: https://issues.apache.org/jira/browse/IGNITE-12914
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Priority: Minor
>
> We need to add next rules to planner
> * AggregateProjectMergeRule
> In order to be able to make this transformation for the query:
> {noformat}
> "select x, sum(z), y from (\n"
> + " select deptno as x, empno as y, sal as z, sal * 2 as zz\n"
> + " from emp)\n"
> + "group by x, y"
> BEFORE=
> LogicalProject(X=[$0], EXPR$1=[$2], Y=[$1])
> LogicalAggregate(group=[{0, 1}], EXPR$1=[SUM($2)])
> LogicalProject(X=[$3], Y=[$0], Z=[$2])
> IgniteTableScan(table=[[PUBLIC, EMP]])
> AFTER=
> IgniteProject(X=[$0], EXPR$1=[$2], Y=[$1])
> IgniteProject(DEPTNO=[$1], EMPNO=[$0], EXPR$1=[$2])
> IgniteAggregate(group=[{0, 3}], EXPR$1=[SUM($2)])
> IgniteTableScan(table=[[PUBLIC, EMP]])
> {noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12913) Calcite integration: Add semi join remove rule to the planner
[
https://issues.apache.org/jira/browse/IGNITE-12913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12913:
Priority: Minor (was: Major)
> Calcite integration: Add semi join remove rule to the planner
> -
>
> Key: IGNITE-12913
> URL: https://issues.apache.org/jira/browse/IGNITE-12913
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Priority: Minor
>
> We need to add next rules to planner
> * FilterJoinRule,
> * JoinAddRedundantSemiJoinRule,
> * SemiJoinRemoveRule
> In order to be able to make this transformation for the query:
> {noformat}
> "select e.ename from emp e, dept d\n"
> + "where e.deptno = d.deptno"
> BEFORE=
> LogicalProject(ENAME=[$1])
> LogicalFilter(condition=[=($3, $5)])
> LogicalJoin(condition=[true], joinType=[inner])
> IgniteTableScan(table=[[PUBLIC, EMP]])
> IgniteTableScan(table=[[PUBLIC, DEPT]])
> AFTER=
> IgniteProject(ENAME=[$1])
> IgniteJoin(condition=[=($3, $5)], joinType=[inner])
> IgniteTableScan(table=[[PUBLIC, EMP]])
> IgniteTableScan(table=[[PUBLIC, DEPT]])
> {noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12633) Calcite integration. Query cancel purpose.
[ https://issues.apache.org/jira/browse/IGNITE-12633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12633: Priority: Minor (was: Major) > Calcite integration. Query cancel purpose. > -- > > Key: IGNITE-12633 > URL: https://issues.apache.org/jira/browse/IGNITE-12633 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Igor Seliverstov >Assignee: Ravil Galeyev >Priority: Minor > > Currently on a query participant node left whole query is cancelled, but the > end user doesn't know the purpose. We should pass causing cancel exception to > end user. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12692) Calcite integration. DML. Skip reducer optimization.
[ https://issues.apache.org/jira/browse/IGNITE-12692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12692: Priority: Minor (was: Major) > Calcite integration. DML. Skip reducer optimization. > > > Key: IGNITE-12692 > URL: https://issues.apache.org/jira/browse/IGNITE-12692 > Project: Ignite > Issue Type: Improvement >Reporter: Igor Seliverstov >Priority: Minor > > Having plan tree we easily can check whether a final modification may be > executed on data nodes directly or not. We should implement such kind of > optimization. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12586) Calcite integration. NodesMapping simplification.
[
https://issues.apache.org/jira/browse/IGNITE-12586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12586:
Priority: Minor (was: Major)
> Calcite integration. NodesMapping simplification.
> -
>
> Key: IGNITE-12586
> URL: https://issues.apache.org/jira/browse/IGNITE-12586
> Project: Ignite
> Issue Type: Improvement
>Reporter: Igor Seliverstov
>Priority: Minor
>
> Seems turning {{List assignments}} to {{Map
> assignments}} may significantly reduce occupied memory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12585) Calcite integration. Splitter improvements.
[ https://issues.apache.org/jira/browse/IGNITE-12585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12585: Priority: Minor (was: Major) > Calcite integration. Splitter improvements. > --- > > Key: IGNITE-12585 > URL: https://issues.apache.org/jira/browse/IGNITE-12585 > Project: Ignite > Issue Type: Improvement >Reporter: Igor Seliverstov >Assignee: Nikolay Izhikov >Priority: Minor > > Currently in case a head fragment of a query plan have broadcast distribution > it causes OptimisticPlanningException and additional fragment split each time > it's mapped on topology on a client node. Seems it's quite usual case, so, we > should cover it properly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13025) Calcite integration. Limit and offset support.
[
https://issues.apache.org/jira/browse/IGNITE-13025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-13025:
Priority: Critical (was: Major)
> Calcite integration. Limit and offset support.
> --
>
> Key: IGNITE-13025
> URL: https://issues.apache.org/jira/browse/IGNITE-13025
> Project: Ignite
> Issue Type: New Feature
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Taras Ledkov
>Priority: Critical
>
> We can exploit {{Sort}} operator for this purpose or introduce a new operator
> {{Limit}} by analogy with {{EnumerableLimit}}. Some research is needed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12455) Calcite integration. Partition pruning.
[ https://issues.apache.org/jira/browse/IGNITE-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12455: Priority: Minor (was: Major) > Calcite integration. Partition pruning. > --- > > Key: IGNITE-12455 > URL: https://issues.apache.org/jira/browse/IGNITE-12455 > Project: Ignite > Issue Type: Improvement >Reporter: Igor Seliverstov >Priority: Minor > > In scope of fragment info calculation we're able to prune partitions on the > basis of filter condition, query parameters, affinity and tables distribution. > This optimization may dramatically reduce query execution time/needed > resources. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12868) Calcite integration. LEFT, RIGHT join support.
[ https://issues.apache.org/jira/browse/IGNITE-12868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12868: Priority: Critical (was: Major) > Calcite integration. LEFT, RIGHT join support. > -- > > Key: IGNITE-12868 > URL: https://issues.apache.org/jira/browse/IGNITE-12868 > Project: Ignite > Issue Type: New Feature >Reporter: Igor Seliverstov >Priority: Critical > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12747) Calcite integration. Correlated queries support.
[ https://issues.apache.org/jira/browse/IGNITE-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12747: Priority: Critical (was: Major) > Calcite integration. Correlated queries support. > > > Key: IGNITE-12747 > URL: https://issues.apache.org/jira/browse/IGNITE-12747 > Project: Ignite > Issue Type: New Feature >Reporter: Igor Seliverstov >Priority: Critical > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13026) Calcite integration. Fix metadata for IgniteTable when it has embedded filters.
[
https://issues.apache.org/jira/browse/IGNITE-13026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-13026:
Priority: Critical (was: Major)
> Calcite integration. Fix metadata for IgniteTable when it has embedded
> filters.
> ---
>
> Key: IGNITE-13026
> URL: https://issues.apache.org/jira/browse/IGNITE-13026
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Konstantin Orlov
>Priority: Critical
>
> By default {{RelMdRowCount}} return the actual row count for
> {{IgniteTables}}. But current implementation of {{IgniteTableScan}} may
> contain embedded filters and rows that are filtered out by these filters are
> not taken into account by the metadata subsystem.
> For example, table emps contains 1000 rows. Filter name='Vasya' filters out
> 95% of rows. For now we end up with situation
> {code:java}
> IgniteFilter(name='Vasya') <- estimated cardinality = 50 rows
> IgniteTableScan(filters=null) <- estimated cardinality = 1000 rows.
> {code}
> but after merging {{Filter}} into {{Scan}} we get the wrong estimation
> {code:java}
> IgniteTableScan(filters={name='Vasya'}) <- estimated cardinality = 1000 rows.
> But should be 50 rows.
> {code}
> We need to add a special case in order to compute the actual cardinality
> returned by {{IgniteTableScan}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13092) ConcurrentModificationException when adding items to cache
[
https://issues.apache.org/jira/browse/IGNITE-13092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119404#comment-17119404
]
Roman Kondakov commented on IGNITE-13092:
-
Please, check [https://apacheignite.readme.io/docs/logging#default-logging]. I
think you need to try to change the logging configuration in
{{$IGNITE_HOME/config/}}. Just try to set at least {{INFO}} debug level in the
next configuration files:
* ignite-log4j.xml
* ignite-log4j2.xml
* java.util.logging.properties
I believe changing the log level in one of this files (depends on the logger
type used) might help you to work the problem around.
> ConcurrentModificationException when adding items to cache
> --
>
> Key: IGNITE-13092
> URL: https://issues.apache.org/jira/browse/IGNITE-13092
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.8.1
> Environment: Java 1.8.0_231
> Apache Ignite 2.8.1
> Windows 10, 64G memory
> Java params:
> {{-Xms1024m }}{{-Xmx50g }}{{-Xss1024m}}
> {{-Xverify:none}}
> {{-server}}
> {{-DIGNITE_QUIET=true}}
> {{-XX:+UseG1GC}}
> {{-XX:+DisableExplicitGC}}
> {{-Djava.net.preferIPv4Stack=true}}
> {{-XX:+AlwaysPreTouch}}
> {{-XX:+ScavengeBeforeFullGC}}
> {{-XX:+AggressiveOpts}}
>
> org.apache.ignite
> ignite-core
> ${ignite.version}
>
>
> org.apache.ignite
> ignite-spring
> ${ignite.version}
>
>
> org.apache.ignite
> ignite-log4j2
> ${ignite.version}
>
>
> http://www.springframework.org/schema/beans;
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd;>
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> class="org.apache.ignite.configuration.DataStorageConfiguration">
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> class="org.apache.ignite.configuration.DataRegionConfiguration">
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>Reporter: Tomasz Grygo
>Priority: Blocker
>
> Ignite fails when adding items to cache with multiple concurrent threads.
> Cache is persistent, with no eviction and no WAL. There is no Ignite cluster,
> just one machine.
> 2020-05-27 23:29:13,177 [Storage0 ] [ERROR] - <.ChildrenStore> Unexpected
> exception during cache update
> java.util.ConcurrentModificationException
> at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1211)
> at java.util.TreeMap$EntryIterator.next(TreeMap.java:1247)
> at java.util.TreeMap$EntryIterator.next(TreeMap.java:1242)
> at java.util.AbstractMap.toString(AbstractMap.java:554)
> at java.lang.String.valueOf(String.java:2994)
> at java.lang.StringBuilder.append(StringBuilder.java:131)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryPartitionRecovery.collectEntries(CacheContinuousQueryPartitionRecovery.java:261)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.handleEvent(CacheContinuousQueryHandler.java:973)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.onEntryUpdate(CacheContinuousQueryHandler.java:1019)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.access$600(CacheContinuousQueryHandler.java:90)
[jira] [Commented] (IGNITE-13092) ConcurrentModificationException when adding items to cache
[
https://issues.apache.org/jira/browse/IGNITE-13092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119050#comment-17119050
]
Roman Kondakov commented on IGNITE-13092:
-
It looks like a bug in the logging: {{TreeMap}} is logged without a proper
synchronization. As a workaround you can set the logging level higher than
{{DEBUG}}
> ConcurrentModificationException when adding items to cache
> --
>
> Key: IGNITE-13092
> URL: https://issues.apache.org/jira/browse/IGNITE-13092
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.8.1
> Environment: Java 1.8.0_231
> Apache Ignite 2.8.1
> Windows 10, 64G memory
> Java params:
> {{-Xms1024m }}{{-Xmx50g }}{{-Xss1024m}}
> {{-Xverify:none}}
> {{-server}}
> {{-DIGNITE_QUIET=true}}
> {{-XX:+UseG1GC}}
> {{-XX:+DisableExplicitGC}}
> {{-Djava.net.preferIPv4Stack=true}}
> {{-XX:+AlwaysPreTouch}}
> {{-XX:+ScavengeBeforeFullGC}}
> {{-XX:+AggressiveOpts}}
>
> org.apache.ignite
> ignite-core
> ${ignite.version}
>
>
> org.apache.ignite
> ignite-spring
> ${ignite.version}
>
>
> org.apache.ignite
> ignite-log4j2
> ${ignite.version}
>
>
> http://www.springframework.org/schema/beans;
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd;>
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> class="org.apache.ignite.configuration.DataStorageConfiguration">
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> class="org.apache.ignite.configuration.DataRegionConfiguration">
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>Reporter: Tomasz Grygo
>Priority: Blocker
>
> Ignite fails when adding items to cache with multiple concurrent threads.
> Cache is persistent, with no eviction and no WAL. There is no Ignite cluster,
> just one machine.
> 2020-05-27 23:29:13,177 [Storage0 ] [ERROR] - <.ChildrenStore> Unexpected
> exception during cache update
> java.util.ConcurrentModificationException
> at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1211)
> at java.util.TreeMap$EntryIterator.next(TreeMap.java:1247)
> at java.util.TreeMap$EntryIterator.next(TreeMap.java:1242)
> at java.util.AbstractMap.toString(AbstractMap.java:554)
> at java.lang.String.valueOf(String.java:2994)
> at java.lang.StringBuilder.append(StringBuilder.java:131)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryPartitionRecovery.collectEntries(CacheContinuousQueryPartitionRecovery.java:261)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.handleEvent(CacheContinuousQueryHandler.java:973)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.onEntryUpdate(CacheContinuousQueryHandler.java:1019)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.access$600(CacheContinuousQueryHandler.java:90)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler$2.onEntryUpdated(CacheContinuousQueryHandler.java:445)
> at
>
[jira] [Commented] (IGNITE-13068) SQL: use cache.localSize instead of index scan to calculate row count statistic
[ https://issues.apache.org/jira/browse/IGNITE-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17117532#comment-17117532 ] Roman Kondakov commented on IGNITE-13068: - [~tledkov-gridgain] patch looks good to me. > SQL: use cache.localSize instead of index scan to calculate row count > statistic > > > Key: IGNITE-13068 > URL: https://issues.apache.org/jira/browse/IGNITE-13068 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Taras Ledkov >Assignee: Taras Ledkov >Priority: Major > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > The scan of index is slow on big data set on setup where disc i/o requires > warm up. > This is the reason that restarting the node can take a long time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13022) Calcite integration. Merge index conditions for the same field.
[
https://issues.apache.org/jira/browse/IGNITE-13022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116140#comment-17116140
]
Roman Kondakov commented on IGNITE-13022:
-
We also need to check that it works for {{BETWEEN}} clause. As well as it
works in the presence of filters over several tables.
> Calcite integration. Merge index conditions for the same field.
> ---
>
> Key: IGNITE-13022
> URL: https://issues.apache.org/jira/browse/IGNITE-13022
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Priority: Major
>
> Index scans should be able to merge index conditions. For example query
> {code:java}
> SELECT * FROM tbl WHERE a<5 AND a<10
> {code}
> should be reduced to
>
> {code:java}
> SELECT * FROM tbl WHERE a<5
> {code}
> Parameters should be handled in a more tricky way:
> {code:java}
> SELECT * FROM tbl WHERE a {code}
> can be rewritten as
> {code:java}
> SELECT * FROM tbl WHERE a where the expression {{MIN(?1, ?2)}} should be evaluated right before the
> execution when parameters are known.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13023) Calcite integration. Page scan support.
[ https://issues.apache.org/jira/browse/IGNITE-13023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116134#comment-17116134 ] Roman Kondakov commented on IGNITE-13023: - It looks like the page scan is implemented poorly. We may skip this issue. > Calcite integration. Page scan support. > --- > > Key: IGNITE-13023 > URL: https://issues.apache.org/jira/browse/IGNITE-13023 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Roman Kondakov >Priority: Major > > We need to evaluate the advantage of using data page scan (see IGNITE-10798) > in Calcite engine and support it if needed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13070) SQL regressions detection framework
Roman Kondakov created IGNITE-13070: --- Summary: SQL regressions detection framework Key: IGNITE-13070 URL: https://issues.apache.org/jira/browse/IGNITE-13070 Project: Ignite Issue Type: Test Components: sql Reporter: Roman Kondakov We need to detect SQL regressions early. We can do it by comparing the SQL query performance for different Ignite versions. This test framework should work in the following way: # It starts two Ignite clusters with different versions (current version and the previous release version). # Framework then runs randomly generated queries in both clusters and checks the execution time for each cluster. We need to port SQLSmith library from C++ to java for this step. But initially we can start with some set of hardcoded queries and postpone the SQLSmith port. Randomized queries can be added later. # All problematic queries are then reported as performance issues. In this way we can manually examine the problems. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13035) Calcite integration. Support fallback on indexes rebuild
Roman Kondakov created IGNITE-13035: --- Summary: Calcite integration. Support fallback on indexes rebuild Key: IGNITE-13035 URL: https://issues.apache.org/jira/browse/IGNITE-13035 Project: Ignite Issue Type: Improvement Components: sql Reporter: Roman Kondakov When indexes are being rebuilt we need to use CacheTree for index scans as it happens in legacy engine. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13030) Calcite integration. Push projections to scans and avoid reading full row when possible
Roman Kondakov created IGNITE-13030:
---
Summary: Calcite integration. Push projections to scans and avoid
reading full row when possible
Key: IGNITE-13030
URL: https://issues.apache.org/jira/browse/IGNITE-13030
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Roman Kondakov
Currently {{IngiteTableScan}} supports only embedded filters. If it supports
also embedded projections (see {{ProjectableFilterableTable}} in Calcite), it
will be able to push these projections down to page memory scans in order to
avoid full row reading.
For example for the query
{code:java}
SELECT _val FROM tbl
{code}
we can read only values.
Moreover, we can establish index-only scans in some cases. For example, if the
column {{depId}} is indexed, we can the execute query
{code:java}
SELECT depId FROM emp
{code}
using only index tree, without reading rows at all.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13027) Calcite integration. Query parallelism support.
Roman Kondakov created IGNITE-13027: --- Summary: Calcite integration. Query parallelism support. Key: IGNITE-13027 URL: https://issues.apache.org/jira/browse/IGNITE-13027 Project: Ignite Issue Type: Improvement Components: sql Reporter: Roman Kondakov We need to add support for segmented indexes in Calcite engine. Currently engine is able to execute queries over index with only one segment. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13026) Calcite integration. Fix metadata for IgniteTable when it has embedded filters.
Roman Kondakov created IGNITE-13026:
---
Summary: Calcite integration. Fix metadata for IgniteTable when it
has embedded filters.
Key: IGNITE-13026
URL: https://issues.apache.org/jira/browse/IGNITE-13026
Project: Ignite
Issue Type: Bug
Components: sql
Reporter: Roman Kondakov
By default {{RelMdRowCount}} return the actual row count for {{IgniteTables}}.
But current implementation of {{IgniteTableScan}} may contain embedded filters
and rows that are filtered out by these filters are not taken into account by
the metadata subsystem.
For example, table emps contains 1000 rows. Filter name='Vasya' filters out
95% of rows. For now we end up with situation
{code:java}
IgniteFilter(name='Vasya') <- estimated cardinality = 50 rows
IgniteTableScan(filters=null) <- estimated cardinality = 1000 rows.
{code}
but after merging {{Filter}} into {{Scan}} we get the wrong estimation
{code:java}
IgniteTableScan(filters={name='Vasya'}) <- estimated cardinality = 1000 rows.
But should be 50 rows.
{code}
We need to add a special case in order to compute the actual cardinality
returned by {{IgniteTableScan}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12909) Calcite integration. Explain command support
[
https://issues.apache.org/jira/browse/IGNITE-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110516#comment-17110516
]
Roman Kondakov commented on IGNITE-12909:
-
Will be fixed in IGNITE-12715. We need check that default Calcite syntax
{{EXPLAIN PLAN FOR ...}} is ok for us.
> Calcite integration. Explain command support
>
>
> Key: IGNITE-12909
> URL: https://issues.apache.org/jira/browse/IGNITE-12909
> Project: Ignite
> Issue Type: Task
>Reporter: Igor Seliverstov
>Priority: Major
>
> Support EXPLAIN command
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13025) Calcite integration. Limit and offset support.
Roman Kondakov created IGNITE-13025:
---
Summary: Calcite integration. Limit and offset support.
Key: IGNITE-13025
URL: https://issues.apache.org/jira/browse/IGNITE-13025
Project: Ignite
Issue Type: New Feature
Components: sql
Reporter: Roman Kondakov
We can exploit {{Sort}} operator for this purpose or introduce a new operator
{{Limit}} by analogy with {{EnumerableLimit}}. Some research is needed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13024) Calcite integration. Support and simplification of complex expressions in index conditions
Roman Kondakov created IGNITE-13024:
---
Summary: Calcite integration. Support and simplification of
complex expressions in index conditions
Key: IGNITE-13024
URL: https://issues.apache.org/jira/browse/IGNITE-13024
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Roman Kondakov
Current implementation supports only trivial expressions in index conditions,
i.e. {{x=1}}. We need to support all evaluatable expressions (which not depends
on input/table references) like {{x=?+10}}. Also we need to ensure that complex
expressions in index filters are simplified (see {{RexSimplify}}).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13021) Calcite integration. Avoid full scans for disjunctive queries.
[
https://issues.apache.org/jira/browse/IGNITE-13021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-13021:
Description:
Currently a full table scan will be executed in the case of disjunctive
predicate even if predicate fields are indexed. For example:
{code:java}
SELECT * FROM emps WHERE name='A' OR surname='B'
{code}
This is caused by the nature of indexes: they can return cursor bounded by
lower and upper bounds. We can cope with it by implementing a logical rule for
rewriting {{OR}} query to a {{UNION ALL}} query:
{code:java}
SELECT * FROM emps WHERE name='A'
UNION ALL
SELECT * FROM emps WHERE surname='B' AND LNNVL(name='A')
{code}
where {{LNNVL()}} function has semantics
{code:java}
LNNVL(name='A') == name!='A' OR name=NULL.
{code}
It is used to avoid expensive deduplication. This name is taken from Oracle, we
can think of more meaningful name, or find the analog in Calcite or H2.
See, for example, this blog post:
[https://blogs.oracle.com/optimizer/optimizer-transformations:-or-expansion]
for details.
Also it is needed to check this works for {{IN}} clause with small number of
literals (AFAIK Calcite converts large {{IN}} clauses to a join with {{Values}}
table where N > 20).
was:
Currently a full table scan will be executed in the case of disjunctive
predicate even if predicate fields are indexed. For example:
{code:java}
SELECT * FROM emps WHERE name='A' OR surname='B'
{code}
This is caused by the nature of indexes: they can return cursor bounded by
lower and upper bounds. We can cope with it by implementing a logical rule for
rewriting {{OR}} query to a {{UNION ALL}} query:
{code:java}
SELECT * FROM emps WHERE name='A'
UNION ALL
SELECT * FROM emps WHERE surname='B' AND LNNVL(name='A')
{code}
where {{LNNVL()}} function has semantics
{code:java}
LNNVL(name='A') == name!='A' OR name=NULL.
{code}
It is used to avoid expensive deduplication. This name is taken from Oracle, we
can think of more meaningful name, or find the analog in Calcite or H2.
See, for example, this blog post:
[https://blogs.oracle.com/optimizer/optimizer-transformations:-or-expansion]
for details.
> Calcite integration. Avoid full scans for disjunctive queries.
> --
>
> Key: IGNITE-13021
> URL: https://issues.apache.org/jira/browse/IGNITE-13021
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Priority: Major
>
> Currently a full table scan will be executed in the case of disjunctive
> predicate even if predicate fields are indexed. For example:
> {code:java}
> SELECT * FROM emps WHERE name='A' OR surname='B'
> {code}
> This is caused by the nature of indexes: they can return cursor bounded by
> lower and upper bounds. We can cope with it by implementing a logical rule
> for rewriting {{OR}} query to a {{UNION ALL}} query:
> {code:java}
> SELECT * FROM emps WHERE name='A'
> UNION ALL
> SELECT * FROM emps WHERE surname='B' AND LNNVL(name='A')
> {code}
> where {{LNNVL()}} function has semantics
> {code:java}
> LNNVL(name='A') == name!='A' OR name=NULL.
> {code}
> It is used to avoid expensive deduplication. This name is taken from Oracle,
> we can think of more meaningful name, or find the analog in Calcite or H2.
> See, for example, this blog post:
> [https://blogs.oracle.com/optimizer/optimizer-transformations:-or-expansion]
> for details.
> Also it is needed to check this works for {{IN}} clause with small number of
> literals (AFAIK Calcite converts large {{IN}} clauses to a join with
> {{Values}} table where N > 20).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13023) Calcite integration. Page scan support.
Roman Kondakov created IGNITE-13023: --- Summary: Calcite integration. Page scan support. Key: IGNITE-13023 URL: https://issues.apache.org/jira/browse/IGNITE-13023 Project: Ignite Issue Type: Improvement Components: sql Reporter: Roman Kondakov We need to evaluate the advantage of using data page scan (see IGNITE-10798) in Calcite engine and support it if needed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13022) Calcite integration. Merge index conditions for the same field.
Roman Kondakov created IGNITE-13022:
---
Summary: Calcite integration. Merge index conditions for the same
field.
Key: IGNITE-13022
URL: https://issues.apache.org/jira/browse/IGNITE-13022
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Roman Kondakov
Index scans should be able to merge index conditions. For example query
{code:java}
SELECT * FROM tbl WHERE a<5 AND a<10
{code}
should be reduced to
{code:java}
SELECT * FROM tbl WHERE a<5
{code}
Parameters should be handled in a more tricky way:
{code:java}
SELECT * FROM tbl WHERE a
[jira] [Created] (IGNITE-13021) Calcite integration. Avoid full scans for disjunctive queries.
Roman Kondakov created IGNITE-13021:
---
Summary: Calcite integration. Avoid full scans for disjunctive
queries.
Key: IGNITE-13021
URL: https://issues.apache.org/jira/browse/IGNITE-13021
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Roman Kondakov
Currently a full table scan will be executed in the case of disjunctive
predicate even if predicate fields are indexed. For example:
{code:java}
SELECT * FROM emps WHERE name='A' OR surname='B'
{code}
This is caused by the nature of indexes: they can return cursor bounded by
lower and upper bounds. We can cope with it by implementing a logical rule for
rewriting {{OR}} query to a {{UNION ALL}} query:
{code:java}
SELECT * FROM emps WHERE name='A'
UNION ALL
SELECT * FROM emps WHERE surname='B' AND LNNVL(name='A')
{code}
where {{LNNVL()}} function has semantics
{code:java}
LNNVL(name='A') == name!='A' OR name=NULL.
{code}
It is used to avoid expensive deduplication. This name is taken from Oracle, we
can think of more meaningful name, or find the analog in Calcite or H2.
See, for example, this blog post:
[https://blogs.oracle.com/optimizer/optimizer-transformations:-or-expansion]
for details.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12912) Calcite integration: Add filters merge rule to the planner
[
https://issues.apache.org/jira/browse/IGNITE-12912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108402#comment-17108402
]
Roman Kondakov commented on IGNITE-12912:
-
[~tledkov-gridgain], looks good to me.
> Calcite integration: Add filters merge rule to the planner
> --
>
> Key: IGNITE-12912
> URL: https://issues.apache.org/jira/browse/IGNITE-12912
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Taras Ledkov
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We need to add next rules to planner
> * FilterMergeRule
> * FilterProjectTransposeRule
> In order to be able to make this transformation for the query:
> {noformat}
> "select name from (\n"
> + " select *\n"
> + " from dept\n"
> + " where deptno = 10)\n"
> + "where deptno = 10\n"
> BEFORE=
> LogicalProject(NAME=[$1])
> LogicalFilter(condition=[=(CAST($0):INTEGER, 10)])
> LogicalProject(DEPTNO=[$0], NAME=[$1])
> LogicalFilter(condition=[=(CAST($0):INTEGER, 10)])
> IgniteTableScan(table=[[PUBLIC, DEPT]])
> AFTER=
> IgniteProject(NAME=[$1])
> IgniteProject(DEPTNO=[$0], NAME=[$1])
> IgniteFilter(condition=[=(CAST($0):INTEGER, 10)])
> IgniteTableScan(table=[[PUBLIC, DEPT]])
> {noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12585) Calcite integration. Splitter improvements.
[ https://issues.apache.org/jira/browse/IGNITE-12585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106308#comment-17106308 ] Roman Kondakov commented on IGNITE-12585: - [~nizhikov] yes it is. > Calcite integration. Splitter improvements. > --- > > Key: IGNITE-12585 > URL: https://issues.apache.org/jira/browse/IGNITE-12585 > Project: Ignite > Issue Type: Improvement >Reporter: Igor Seliverstov >Assignee: Nikolay Izhikov >Priority: Major > > Currently in case a head fragment of a query plan have broadcast distribution > it causes OptimisticPlanningException and additional fragment split each time > it's mapped on topology on a client node. Seems it's quite usual case, so, we > should cover it properly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12732) SQL: KILL QUERY command hangs on query when query cursor is held by user or leak
[ https://issues.apache.org/jira/browse/IGNITE-12732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089544#comment-17089544 ] Roman Kondakov commented on IGNITE-12732: - [~nizhikov], here it is: [https://github.com/apache/ignite/pull/7706] > SQL: KILL QUERY command hangs on query when query cursor is held by user or > leak > > > Key: IGNITE-12732 > URL: https://issues.apache.org/jira/browse/IGNITE-12732 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Roman Kondakov >Assignee: Roman Kondakov >Priority: Major > Fix For: 2.9, 2.8.1 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Scenario to reproduce: > 1. Execute a query > 2. Don't close the query cursor, don't iterate to the end of results. > 3. Run "KILL QUERY" command. The command hangs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12732) SQL: KILL QUERY command hangs on query when query cursor is held by user or leak
[ https://issues.apache.org/jira/browse/IGNITE-12732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088513#comment-17088513 ] Roman Kondakov commented on IGNITE-12732: - [~nizhikov], I'm not a committer, so I don't have rights for this. > SQL: KILL QUERY command hangs on query when query cursor is held by user or > leak > > > Key: IGNITE-12732 > URL: https://issues.apache.org/jira/browse/IGNITE-12732 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Roman Kondakov >Assignee: Roman Kondakov >Priority: Major > Fix For: 2.9, 2.8.1 > > Time Spent: 20m > Remaining Estimate: 0h > > Scenario to reproduce: > 1. Execute a query > 2. Don't close the query cursor, don't iterate to the end of results. > 3. Run "KILL QUERY" command. The command hangs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12915) Calcite integration: Add push filter to join rule to the planner
[
https://issues.apache.org/jira/browse/IGNITE-12915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12915:
Description:
We need to add next rules to planner
* FilterJoinRule
* JoinPushExpressionsRule
In order to be able to make this transformation for the query:
{noformat}
"select d.deptno, e.deptno from sales.dept d, sales.emp e\n"
+ " where d.deptno + 10 = e.deptno * 2"
BEFORE=
LogicalProject(DEPTNO=[$0], DEPTNO0=[$5])
LogicalFilter(condition=[=(+($0, 10), *($5, 2))])
LogicalJoin(condition=[true], joinType=[inner])
IgniteTableScan(table=[[PUBLIC, DEPT]])
IgniteTableScan(table=[[PUBLIC, EMP]])
AFTER=
IgniteProject(DEPTNO=[$0], DEPTNO0=[$5])
IgniteJoin(condition=[=(+($0, 10), *($5, 2))], joinType=[inner])
IgniteTableScan(table=[[PUBLIC, DEPT]])
IgniteTableScan(table=[[PUBLIC, EMP]])
{noformat}
was:
We need to add next rules to planner
* AggregateProjectMergeRule
In order to be able to make this transformation for the query:
{noformat}
"select x, sum(z), y from (\n"
+ " select deptno as x, empno as y, sal as z, sal * 2 as zz\n"
+ " from emp)\n"
+ "group by x, y"
BEFORE=
LogicalProject(X=[$0], EXPR$1=[$2], Y=[$1])
LogicalAggregate(group=[{0, 1}], EXPR$1=[SUM($2)])
LogicalProject(X=[$3], Y=[$0], Z=[$2])
IgniteTableScan(table=[[PUBLIC, EMP]])
AFTER=
IgniteProject(X=[$0], EXPR$1=[$2], Y=[$1])
IgniteProject(DEPTNO=[$1], EMPNO=[$0], EXPR$1=[$2])
IgniteAggregate(group=[{0, 3}], EXPR$1=[SUM($2)])
IgniteTableScan(table=[[PUBLIC, EMP]])
{noformat}
> Calcite integration: Add push filter to join rule to the planner
>
>
> Key: IGNITE-12915
> URL: https://issues.apache.org/jira/browse/IGNITE-12915
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Priority: Major
>
> We need to add next rules to planner
> * FilterJoinRule
> * JoinPushExpressionsRule
> In order to be able to make this transformation for the query:
> {noformat}
> "select d.deptno, e.deptno from sales.dept d, sales.emp e\n"
> + " where d.deptno + 10 = e.deptno * 2"
> BEFORE=
> LogicalProject(DEPTNO=[$0], DEPTNO0=[$5])
> LogicalFilter(condition=[=(+($0, 10), *($5, 2))])
> LogicalJoin(condition=[true], joinType=[inner])
> IgniteTableScan(table=[[PUBLIC, DEPT]])
> IgniteTableScan(table=[[PUBLIC, EMP]])
> AFTER=
> IgniteProject(DEPTNO=[$0], DEPTNO0=[$5])
> IgniteJoin(condition=[=(+($0, 10), *($5, 2))], joinType=[inner])
> IgniteTableScan(table=[[PUBLIC, DEPT]])
> IgniteTableScan(table=[[PUBLIC, EMP]])
> {noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12915) Calcite integration: Add push filter to join rule to the planner
Roman Kondakov created IGNITE-12915:
---
Summary: Calcite integration: Add push filter to join rule to the
planner
Key: IGNITE-12915
URL: https://issues.apache.org/jira/browse/IGNITE-12915
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Roman Kondakov
We need to add next rules to planner
* AggregateProjectMergeRule
In order to be able to make this transformation for the query:
{noformat}
"select x, sum(z), y from (\n"
+ " select deptno as x, empno as y, sal as z, sal * 2 as zz\n"
+ " from emp)\n"
+ "group by x, y"
BEFORE=
LogicalProject(X=[$0], EXPR$1=[$2], Y=[$1])
LogicalAggregate(group=[{0, 1}], EXPR$1=[SUM($2)])
LogicalProject(X=[$3], Y=[$0], Z=[$2])
IgniteTableScan(table=[[PUBLIC, EMP]])
AFTER=
IgniteProject(X=[$0], EXPR$1=[$2], Y=[$1])
IgniteProject(DEPTNO=[$1], EMPNO=[$0], EXPR$1=[$2])
IgniteAggregate(group=[{0, 3}], EXPR$1=[SUM($2)])
IgniteTableScan(table=[[PUBLIC, EMP]])
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12914) Calcite integration: Add aggregate project merge rule to the planner
[
https://issues.apache.org/jira/browse/IGNITE-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12914:
Description:
We need to add next rules to planner
* AggregateProjectMergeRule
In order to be able to make this transformation for the query:
{noformat}
"select x, sum(z), y from (\n"
+ " select deptno as x, empno as y, sal as z, sal * 2 as zz\n"
+ " from emp)\n"
+ "group by x, y"
BEFORE=
LogicalProject(X=[$0], EXPR$1=[$2], Y=[$1])
LogicalAggregate(group=[{0, 1}], EXPR$1=[SUM($2)])
LogicalProject(X=[$3], Y=[$0], Z=[$2])
IgniteTableScan(table=[[PUBLIC, EMP]])
AFTER=
IgniteProject(X=[$0], EXPR$1=[$2], Y=[$1])
IgniteProject(DEPTNO=[$1], EMPNO=[$0], EXPR$1=[$2])
IgniteAggregate(group=[{0, 3}], EXPR$1=[SUM($2)])
IgniteTableScan(table=[[PUBLIC, EMP]])
{noformat}
was:
We need to add next rules to planner
* FilterJoinRule,
* JoinAddRedundantSemiJoinRule,
* SemiJoinRemoveRule
In order to be able to make this transformation for the query:
{noformat}
"select e.ename from emp e, dept d\n"
+ "where e.deptno = d.deptno"
BEFORE=
LogicalProject(ENAME=[$1])
LogicalFilter(condition=[=($3, $5)])
LogicalJoin(condition=[true], joinType=[inner])
IgniteTableScan(table=[[PUBLIC, EMP]])
IgniteTableScan(table=[[PUBLIC, DEPT]])
AFTER=
IgniteProject(ENAME=[$1])
IgniteJoin(condition=[=($3, $5)], joinType=[inner])
IgniteTableScan(table=[[PUBLIC, EMP]])
IgniteTableScan(table=[[PUBLIC, DEPT]])
{noformat}
> Calcite integration: Add aggregate project merge rule to the planner
>
>
> Key: IGNITE-12914
> URL: https://issues.apache.org/jira/browse/IGNITE-12914
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Priority: Major
>
> We need to add next rules to planner
> * AggregateProjectMergeRule
> In order to be able to make this transformation for the query:
> {noformat}
> "select x, sum(z), y from (\n"
> + " select deptno as x, empno as y, sal as z, sal * 2 as zz\n"
> + " from emp)\n"
> + "group by x, y"
> BEFORE=
> LogicalProject(X=[$0], EXPR$1=[$2], Y=[$1])
> LogicalAggregate(group=[{0, 1}], EXPR$1=[SUM($2)])
> LogicalProject(X=[$3], Y=[$0], Z=[$2])
> IgniteTableScan(table=[[PUBLIC, EMP]])
> AFTER=
> IgniteProject(X=[$0], EXPR$1=[$2], Y=[$1])
> IgniteProject(DEPTNO=[$1], EMPNO=[$0], EXPR$1=[$2])
> IgniteAggregate(group=[{0, 3}], EXPR$1=[SUM($2)])
> IgniteTableScan(table=[[PUBLIC, EMP]])
> {noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12914) Calcite integration: Add aggregate project merge rule to the planner
Roman Kondakov created IGNITE-12914:
---
Summary: Calcite integration: Add aggregate project merge rule to
the planner
Key: IGNITE-12914
URL: https://issues.apache.org/jira/browse/IGNITE-12914
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Roman Kondakov
We need to add next rules to planner
* FilterJoinRule,
* JoinAddRedundantSemiJoinRule,
* SemiJoinRemoveRule
In order to be able to make this transformation for the query:
{noformat}
"select e.ename from emp e, dept d\n"
+ "where e.deptno = d.deptno"
BEFORE=
LogicalProject(ENAME=[$1])
LogicalFilter(condition=[=($3, $5)])
LogicalJoin(condition=[true], joinType=[inner])
IgniteTableScan(table=[[PUBLIC, EMP]])
IgniteTableScan(table=[[PUBLIC, DEPT]])
AFTER=
IgniteProject(ENAME=[$1])
IgniteJoin(condition=[=($3, $5)], joinType=[inner])
IgniteTableScan(table=[[PUBLIC, EMP]])
IgniteTableScan(table=[[PUBLIC, DEPT]])
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12913) Calcite integration: Add semi join remove rule to the planner
[
https://issues.apache.org/jira/browse/IGNITE-12913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12913:
Description:
We need to add next rules to planner
* FilterJoinRule,
* JoinAddRedundantSemiJoinRule,
* SemiJoinRemoveRule
In order to be able to make this transformation for the query:
{noformat}
"select e.ename from emp e, dept d\n"
+ "where e.deptno = d.deptno"
BEFORE=
LogicalProject(ENAME=[$1])
LogicalFilter(condition=[=($3, $5)])
LogicalJoin(condition=[true], joinType=[inner])
IgniteTableScan(table=[[PUBLIC, EMP]])
IgniteTableScan(table=[[PUBLIC, DEPT]])
AFTER=
IgniteProject(ENAME=[$1])
IgniteJoin(condition=[=($3, $5)], joinType=[inner])
IgniteTableScan(table=[[PUBLIC, EMP]])
IgniteTableScan(table=[[PUBLIC, DEPT]])
{noformat}
was:
We need to add next rules to planner
* FilterMergeRule
* FilterProjectTransposeRule
In order to be able to make this transformation for the query:
{noformat}
"select name from (\n"
+ " select *\n"
+ " from dept\n"
+ " where deptno = 10)\n"
+ "where deptno = 10\n"
BEFORE=
LogicalProject(NAME=[$1])
LogicalFilter(condition=[=(CAST($0):INTEGER, 10)])
LogicalProject(DEPTNO=[$0], NAME=[$1])
LogicalFilter(condition=[=(CAST($0):INTEGER, 10)])
IgniteTableScan(table=[[PUBLIC, DEPT]])
AFTER=
IgniteProject(NAME=[$1])
IgniteProject(DEPTNO=[$0], NAME=[$1])
IgniteFilter(condition=[=(CAST($0):INTEGER, 10)])
IgniteTableScan(table=[[PUBLIC, DEPT]])
{noformat}
> Calcite integration: Add semi join remove rule to the planner
> -
>
> Key: IGNITE-12913
> URL: https://issues.apache.org/jira/browse/IGNITE-12913
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Priority: Major
>
> We need to add next rules to planner
> * FilterJoinRule,
> * JoinAddRedundantSemiJoinRule,
> * SemiJoinRemoveRule
> In order to be able to make this transformation for the query:
> {noformat}
> "select e.ename from emp e, dept d\n"
> + "where e.deptno = d.deptno"
> BEFORE=
> LogicalProject(ENAME=[$1])
> LogicalFilter(condition=[=($3, $5)])
> LogicalJoin(condition=[true], joinType=[inner])
> IgniteTableScan(table=[[PUBLIC, EMP]])
> IgniteTableScan(table=[[PUBLIC, DEPT]])
> AFTER=
> IgniteProject(ENAME=[$1])
> IgniteJoin(condition=[=($3, $5)], joinType=[inner])
> IgniteTableScan(table=[[PUBLIC, EMP]])
> IgniteTableScan(table=[[PUBLIC, DEPT]])
> {noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12913) Calcite integration: Add semi join remove rule to the planner
Roman Kondakov created IGNITE-12913:
---
Summary: Calcite integration: Add semi join remove rule to the
planner
Key: IGNITE-12913
URL: https://issues.apache.org/jira/browse/IGNITE-12913
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Roman Kondakov
We need to add next rules to planner
* FilterMergeRule
* FilterProjectTransposeRule
In order to be able to make this transformation for the query:
{noformat}
"select name from (\n"
+ " select *\n"
+ " from dept\n"
+ " where deptno = 10)\n"
+ "where deptno = 10\n"
BEFORE=
LogicalProject(NAME=[$1])
LogicalFilter(condition=[=(CAST($0):INTEGER, 10)])
LogicalProject(DEPTNO=[$0], NAME=[$1])
LogicalFilter(condition=[=(CAST($0):INTEGER, 10)])
IgniteTableScan(table=[[PUBLIC, DEPT]])
AFTER=
IgniteProject(NAME=[$1])
IgniteProject(DEPTNO=[$0], NAME=[$1])
IgniteFilter(condition=[=(CAST($0):INTEGER, 10)])
IgniteTableScan(table=[[PUBLIC, DEPT]])
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12912) Calcite integration: Add filters merge rule to the planner
[
https://issues.apache.org/jira/browse/IGNITE-12912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12912:
Summary: Calcite integration: Add filters merge rule to the planner (was:
Calcite integration: Add filters merge rule to plannner)
> Calcite integration: Add filters merge rule to the planner
> --
>
> Key: IGNITE-12912
> URL: https://issues.apache.org/jira/browse/IGNITE-12912
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Priority: Major
>
> We need to add next rules to planner
> * FilterMergeRule
> * FilterProjectTransposeRule
> In order to be able to make this transformation for the query:
> {noformat}
> "select name from (\n"
> + " select *\n"
> + " from dept\n"
> + " where deptno = 10)\n"
> + "where deptno = 10\n"
> BEFORE=
> LogicalProject(NAME=[$1])
> LogicalFilter(condition=[=(CAST($0):INTEGER, 10)])
> LogicalProject(DEPTNO=[$0], NAME=[$1])
> LogicalFilter(condition=[=(CAST($0):INTEGER, 10)])
> IgniteTableScan(table=[[PUBLIC, DEPT]])
> AFTER=
> IgniteProject(NAME=[$1])
> IgniteProject(DEPTNO=[$0], NAME=[$1])
> IgniteFilter(condition=[=(CAST($0):INTEGER, 10)])
> IgniteTableScan(table=[[PUBLIC, DEPT]])
> {noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12912) Calcite integration: Add filters merge rule to plannner
Roman Kondakov created IGNITE-12912:
---
Summary: Calcite integration: Add filters merge rule to plannner
Key: IGNITE-12912
URL: https://issues.apache.org/jira/browse/IGNITE-12912
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Roman Kondakov
We need to add next rules to planner
* FilterMergeRule
* FilterProjectTransposeRule
In order to be able to make this transformation for the query:
{noformat}
"select name from (\n"
+ " select *\n"
+ " from dept\n"
+ " where deptno = 10)\n"
+ "where deptno = 10\n"
BEFORE=
LogicalProject(NAME=[$1])
LogicalFilter(condition=[=(CAST($0):INTEGER, 10)])
LogicalProject(DEPTNO=[$0], NAME=[$1])
LogicalFilter(condition=[=(CAST($0):INTEGER, 10)])
IgniteTableScan(table=[[PUBLIC, DEPT]])
AFTER=
IgniteProject(NAME=[$1])
IgniteProject(DEPTNO=[$0], NAME=[$1])
IgniteFilter(condition=[=(CAST($0):INTEGER, 10)])
IgniteTableScan(table=[[PUBLIC, DEPT]])
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12852) Comma in field is not supported by COPY command
[ https://issues.apache.org/jira/browse/IGNITE-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085560#comment-17085560 ] Roman Kondakov commented on IGNITE-12852: - [~nizhikov] the patch looks good to me. > Comma in field is not supported by COPY command > --- > > Key: IGNITE-12852 > URL: https://issues.apache.org/jira/browse/IGNITE-12852 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 2.8 >Reporter: YuJue Li >Assignee: YuJue Li >Priority: Critical > Fix For: 2.8.1 > > Time Spent: 10m > Remaining Estimate: 0h > > CREATE TABLE test(a int,b varchar(100),c int,PRIMARY key(a)); > > a.csv: > 1,"a,b",2 > > COPY FROM '/data/a.csv' INTO test (a,b,c) FORMAT CSV; > > The copy command fails because there is a comma in the second field,but this > is a fully legal and compliant CSV format -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12632) [IEP-39] Management API to cancel user provided tasks and queries.
[ https://issues.apache.org/jira/browse/IGNITE-12632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17060882#comment-17060882 ] Roman Kondakov commented on IGNITE-12632: - [~nizhikov], I've left a couple comments in PR. > [IEP-39] Management API to cancel user provided tasks and queries. > -- > > Key: IGNITE-12632 > URL: https://issues.apache.org/jira/browse/IGNITE-12632 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-39 > Time Spent: 1h 10m > Remaining Estimate: 0h > > Ignite provides many API to deploy and execute user-provided code on the > server nodes inside the same JVM as the Ignite process runs. > Ignite has many APIs that allocate many resources on the server nodes, also. > In case of some buggy code that consumes many system resources(CPU, RAM, > flood network) or heavy query the whole cluster can become unstable. > We should provide to the cluster administrator the ability to stop any user > deployed task. > JMX beans to cancel listed tasks should be introduced: > * Compute task > * Service > * Continuous query > * Transactions > * Queries(SQL, Scan, Text) > In the scope of IEP-35 System view API introduced. > A new API should use the same identifier that is used in corresponding System > View. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12771) Java Doc: Marshaller: behavior of "marshal" method if null parameter passed
[
https://issues.apache.org/jira/browse/IGNITE-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057137#comment-17057137
]
Roman Kondakov commented on IGNITE-12771:
-
It looks like the javadoc suite is OK on
[TC|https://ci.ignite.apache.org/project/IgniteTests24Java8?branch=pull%2F7512%2Fhead=overview=builds#all-projects].
> Java Doc: Marshaller: behavior of "marshal" method if null parameter passed
> ---
>
> Key: IGNITE-12771
> URL: https://issues.apache.org/jira/browse/IGNITE-12771
> Project: Ignite
> Issue Type: Bug
>Reporter: Roman Kondakov
>Priority: Trivial
> Time Spent: 10m
> Remaining Estimate: 0h
>
> After fix for GG-19786 java docs looks like this:
> {code:java}
> marshal
> void marshal(@Nullable
> @Nullable Object obj,
> OutputStream out)
> throws IgniteCheckedException
> Marshals object to the output stream. This method should not close given
> output stream. Note: null obj will be marshaled to binary null representation.
> Parameters:
> obj - Object to marshal.
> out - Output stream to marshal into.
> Throws:
> IgniteCheckedException - If marshalling failed.
> marshal
> byte[] marshal(@Nullable
>@Nullable Object obj)
> throws IgniteCheckedException
> Marshals object to byte array. Note: null obj will be marshaled to binary
> null representation.
> Parameters:
> obj - Object to marshal.
> Returns:
> Byte array.
> Throws:
> IgniteCheckedException - If marshalling failed.
> unmarshal
> T unmarshal(InputStream in,
> @Nullable
> @Nullable ClassLoader clsLdr)
> throws IgniteCheckedException
> Unmarshals object from the input stream using given class loader. This method
> should not close given input stream.
> Type Parameters:
> T - Type of unmarshalled object.
> Parameters:
> in - Input stream.
> clsLdr - Class loader to use.
> Returns:
> Unmarshalled object.
> Throws:
> IgniteCheckedException - If unmarshalling failed.
> unmarshal
> T unmarshal(byte[] arr,
> @Nullable
> @Nullable ClassLoader clsLdr)
> throws IgniteCheckedException
> Unmarshals object from byte array using given class loader.
> Type Parameters:
> T - Type of unmarshalled object.
> Parameters:
> arr - Byte array.
> clsLdr - Class loader to use.
> Returns:
> Unmarshalled object.
> Throws:
> IgniteCheckedException - If unmarshalling failed.
> {code}
> 1. Doubles for @Nullable
> 2. question what's the point of allowing null value? is not addressed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12771) Java Doc: Marshaller: behavior of "marshal" method if null parameter passed
[
https://issues.apache.org/jira/browse/IGNITE-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057103#comment-17057103
]
Roman Kondakov commented on IGNITE-12771:
-
[~amashenkov], please take a look. This is a minor javadoc patch.
> Java Doc: Marshaller: behavior of "marshal" method if null parameter passed
> ---
>
> Key: IGNITE-12771
> URL: https://issues.apache.org/jira/browse/IGNITE-12771
> Project: Ignite
> Issue Type: Bug
>Reporter: Roman Kondakov
>Priority: Trivial
> Time Spent: 10m
> Remaining Estimate: 0h
>
> After fix for GG-19786 java docs looks like this:
> {code:java}
> marshal
> void marshal(@Nullable
> @Nullable Object obj,
> OutputStream out)
> throws IgniteCheckedException
> Marshals object to the output stream. This method should not close given
> output stream. Note: null obj will be marshaled to binary null representation.
> Parameters:
> obj - Object to marshal.
> out - Output stream to marshal into.
> Throws:
> IgniteCheckedException - If marshalling failed.
> marshal
> byte[] marshal(@Nullable
>@Nullable Object obj)
> throws IgniteCheckedException
> Marshals object to byte array. Note: null obj will be marshaled to binary
> null representation.
> Parameters:
> obj - Object to marshal.
> Returns:
> Byte array.
> Throws:
> IgniteCheckedException - If marshalling failed.
> unmarshal
> T unmarshal(InputStream in,
> @Nullable
> @Nullable ClassLoader clsLdr)
> throws IgniteCheckedException
> Unmarshals object from the input stream using given class loader. This method
> should not close given input stream.
> Type Parameters:
> T - Type of unmarshalled object.
> Parameters:
> in - Input stream.
> clsLdr - Class loader to use.
> Returns:
> Unmarshalled object.
> Throws:
> IgniteCheckedException - If unmarshalling failed.
> unmarshal
> T unmarshal(byte[] arr,
> @Nullable
> @Nullable ClassLoader clsLdr)
> throws IgniteCheckedException
> Unmarshals object from byte array using given class loader.
> Type Parameters:
> T - Type of unmarshalled object.
> Parameters:
> arr - Byte array.
> clsLdr - Class loader to use.
> Returns:
> Unmarshalled object.
> Throws:
> IgniteCheckedException - If unmarshalling failed.
> {code}
> 1. Doubles for @Nullable
> 2. question what's the point of allowing null value? is not addressed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12771) Java Doc: Marshaller: behavior of "marshal" method if null parameter passed
Roman Kondakov created IGNITE-12771:
---
Summary: Java Doc: Marshaller: behavior of "marshal" method if
null parameter passed
Key: IGNITE-12771
URL: https://issues.apache.org/jira/browse/IGNITE-12771
Project: Ignite
Issue Type: Bug
Reporter: Roman Kondakov
After fix for GG-19786 java docs looks like this:
{code:java}
marshal
void marshal(@Nullable
@Nullable Object obj,
OutputStream out)
throws IgniteCheckedException
Marshals object to the output stream. This method should not close given output
stream. Note: null obj will be marshaled to binary null representation.
Parameters:
obj - Object to marshal.
out - Output stream to marshal into.
Throws:
IgniteCheckedException - If marshalling failed.
marshal
byte[] marshal(@Nullable
@Nullable Object obj)
throws IgniteCheckedException
Marshals object to byte array. Note: null obj will be marshaled to binary null
representation.
Parameters:
obj - Object to marshal.
Returns:
Byte array.
Throws:
IgniteCheckedException - If marshalling failed.
unmarshal
T unmarshal(InputStream in,
@Nullable
@Nullable ClassLoader clsLdr)
throws IgniteCheckedException
Unmarshals object from the input stream using given class loader. This method
should not close given input stream.
Type Parameters:
T - Type of unmarshalled object.
Parameters:
in - Input stream.
clsLdr - Class loader to use.
Returns:
Unmarshalled object.
Throws:
IgniteCheckedException - If unmarshalling failed.
unmarshal
T unmarshal(byte[] arr,
@Nullable
@Nullable ClassLoader clsLdr)
throws IgniteCheckedException
Unmarshals object from byte array using given class loader.
Type Parameters:
T - Type of unmarshalled object.
Parameters:
arr - Byte array.
clsLdr - Class loader to use.
Returns:
Unmarshalled object.
Throws:
IgniteCheckedException - If unmarshalling failed.
{code}
1. Doubles for @Nullable
2. question what's the point of allowing null value? is not addressed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12708) Calcite integration. Expressions factory base implementation.
[ https://issues.apache.org/jira/browse/IGNITE-12708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17051312#comment-17051312 ] Roman Kondakov commented on IGNITE-12708: - [~gvvinblade], I've left couple comments in PR > Calcite integration. Expressions factory base implementation. > - > > Key: IGNITE-12708 > URL: https://issues.apache.org/jira/browse/IGNITE-12708 > Project: Ignite > Issue Type: New Feature > Components: sql >Reporter: Igor Seliverstov >Assignee: Igor Seliverstov >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > We need to implement basic environment for expressions evaluation. > Expressions should exist in two types - compiled and interpreted. > Expressions should be compiled in dedicated thread pull and eventually > replace interpreted ones. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-12732) SQL: KILL QUERY command hangs on query when query cursor is held by user or leak
[ https://issues.apache.org/jira/browse/IGNITE-12732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov reassigned IGNITE-12732: --- Assignee: Roman Kondakov > SQL: KILL QUERY command hangs on query when query cursor is held by user or > leak > > > Key: IGNITE-12732 > URL: https://issues.apache.org/jira/browse/IGNITE-12732 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Roman Kondakov >Assignee: Roman Kondakov >Priority: Major > > Scenario to reproduce: > 1. Execute a query > 2. Don't close the query cursor, don't iterate to the end of results. > 3. Run "KILL QUERY" command. The command hangs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12732) SQL: KILL QUERY command hangs on query when query cursor is held by user or leak
Roman Kondakov created IGNITE-12732: --- Summary: SQL: KILL QUERY command hangs on query when query cursor is held by user or leak Key: IGNITE-12732 URL: https://issues.apache.org/jira/browse/IGNITE-12732 Project: Ignite Issue Type: Bug Components: sql Reporter: Roman Kondakov Scenario to reproduce: 1. Execute a query 2. Don't close the query cursor, don't iterate to the end of results. 3. Run "KILL QUERY" command. The command hangs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12715) Calcite integration. Secondary indexes support.
[ https://issues.apache.org/jira/browse/IGNITE-12715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042902#comment-17042902 ] Roman Kondakov commented on IGNITE-12715: - [~Pavlukhin], yes, this is the most natural way to add indexes into the Calcite planner. > Calcite integration. Secondary indexes support. > --- > > Key: IGNITE-12715 > URL: https://issues.apache.org/jira/browse/IGNITE-12715 > Project: Ignite > Issue Type: New Feature > Components: sql >Reporter: Roman Kondakov >Assignee: Roman Kondakov >Priority: Major > > Secondary indexes should be supported by Calcite-based engine as well as they > supported by the legacy H2 engine. At first we can use the old engine for > indexes maintenance (building, updating, etc). In this case Calcite engine > will only use indexes metadata for query planning and index scans for query > execution. On the next iteration we need to eliminate the old engine usage. > Approximate plan for indexes support implementation: > # Add indexes to schema and set up all schema listeners. > # Add Collation to planner's output trait set and check if generated plan is > properly chosen in accordance to index sort direction. > # Implement index scans with filtering > # Add Sort node to exec. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-12715) Calcite integration. Secondary indexes support.
[ https://issues.apache.org/jira/browse/IGNITE-12715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov reassigned IGNITE-12715: --- Assignee: Roman Kondakov > Calcite integration. Secondary indexes support. > --- > > Key: IGNITE-12715 > URL: https://issues.apache.org/jira/browse/IGNITE-12715 > Project: Ignite > Issue Type: New Feature > Components: sql >Reporter: Roman Kondakov >Assignee: Roman Kondakov >Priority: Major > > Secondary indexes should be supported by Calcite-based engine as well as they > supported by the legacy H2 engine. At first we can use the old engine for > indexes maintenance (building, updating, etc). In this case Calcite engine > will only use indexes metadata for query planning and index scans for query > execution. On the next iteration we need to eliminate the old engine usage. > Approximate plan for indexes support implementation: > # Add indexes to schema and set up all schema listeners. > # Add Collation to planner's output trait set and check if generated plan is > properly chosen in accordance to index sort direction. > # Implement index scans with filtering > # Add Sort node to exec. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12715) Calcite integration. Secondary indexes support.
Roman Kondakov created IGNITE-12715: --- Summary: Calcite integration. Secondary indexes support. Key: IGNITE-12715 URL: https://issues.apache.org/jira/browse/IGNITE-12715 Project: Ignite Issue Type: New Feature Components: sql Reporter: Roman Kondakov Secondary indexes should be supported by Calcite-based engine as well as they supported by the legacy H2 engine. At first we can use the old engine for indexes maintenance (building, updating, etc). In this case Calcite engine will only use indexes metadata for query planning and index scans for query execution. On the next iteration we need to eliminate the old engine usage. Approximate plan for indexes support implementation: # Add indexes to schema and set up all schema listeners. # Add Collation to planner's output trait set and check if generated plan is properly chosen in accordance to index sort direction. # Implement index scans with filtering # Add Sort node to exec. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12602) Calcite integration. JDBC Thin driver support.
[ https://issues.apache.org/jira/browse/IGNITE-12602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035329#comment-17035329 ] Roman Kondakov commented on IGNITE-12602: - [~gvvinblade], patch looks good for me. I've left a couple minor comments in PR. > Calcite integration. JDBC Thin driver support. > -- > > Key: IGNITE-12602 > URL: https://issues.apache.org/jira/browse/IGNITE-12602 > Project: Ignite > Issue Type: Task >Reporter: Igor Seliverstov >Assignee: Igor Seliverstov >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > Provide a way to use experimental engine via JDBC thin driver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12448) Calcite integration. Communication protocol.
[ https://issues.apache.org/jira/browse/IGNITE-12448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023780#comment-17023780 ] Roman Kondakov commented on IGNITE-12448: - [~gvvinblade], overall patch looks good for me. I left several minor comments in PR. > Calcite integration. Communication protocol. > > > Key: IGNITE-12448 > URL: https://issues.apache.org/jira/browse/IGNITE-12448 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Igor Seliverstov >Assignee: Igor Seliverstov >Priority: Major > Time Spent: 2.5h > Remaining Estimate: 0h > > We need to introduce a communication protocol between query fragments in > scope of query execution. > * Protocol should have Push semantics with back pressure ability > * Protocol have to provide ordering guaranties - the data batches processed > in same order they sent to remote node. > * Protocol have to provide delivery guaranties. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12446) Calcite integration. Fix javadocs and code style.
[ https://issues.apache.org/jira/browse/IGNITE-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003323#comment-17003323 ] Roman Kondakov commented on IGNITE-12446: - [~gvvinblade], I've added a couple comments on PR, please take a look. > Calcite integration. Fix javadocs and code style. > - > > Key: IGNITE-12446 > URL: https://issues.apache.org/jira/browse/IGNITE-12446 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Igor Seliverstov >Assignee: Igor Seliverstov >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > Fix javadocs and code style -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12248) Apache Calcite based query execution engine
[
https://issues.apache.org/jira/browse/IGNITE-12248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981405#comment-16981405
]
Roman Kondakov commented on IGNITE-12248:
-
[~gvvinblade], I've taken a look to your approach and overall it seems to be
very good.
I just have a list of minor questions:
# Why do we need spring dependencies in calcite/pom.xml?
# Are we going to support {{_key}} and {{_val}} columns in the new engine? Why?
# Why do we use java serialization in calcite.serialize package?
# What is the purpose of {{CalciteSchemaHolder}}? Why don't use just
{{RootSchema}}? I haven't got the idea.
# Why do we need {{IgnitePlanner}}? It is almost certain copy of
{{org.apache.calcite.prepare.PlannerImpl}}
# Why do we wrap everything into {{Context}}? Even query itself. Do we really
need it? See {{CalciteQueryProcessor#context()}}. Excessive use of
wrap/unwrap/chaining/etc methods affects code readablity.
# Looks like {{RelOp}} interface is redundant. Also as usage of {{BiFunction}}
in {{Commons#transformSubset}}. IMO we need to avoid excessive using of lambdas
and functions. It affects readability too.
# Why do we need to have {{DistributionRegistry}}? Aren't tables aware of
their distributions?
# IgniteMdDistribution#join - there is a list of differnet types of joins and
there is also a statement that "others are impossible". Why broadcast LEFT JOIN
hash is impossible?
# What is the difference between implementor and visitor? See
{{RelImplementor}}. If there are no differences, why don't use the word
{{Visitor}} instead of {{Implementor}} just because it is the well known
pattern?
> Apache Calcite based query execution engine
> ---
>
> Key: IGNITE-12248
> URL: https://issues.apache.org/jira/browse/IGNITE-12248
> Project: Ignite
> Issue Type: Task
> Components: sql
>Reporter: Igor Seliverstov
>Assignee: Igor Seliverstov
>Priority: Major
>
> Currently used H2 based query execution engine has a number of critical
> limitations Which do not allow to execute an arbitrary query.
> The ticket aims to show potential of a new, Calcite based, execution engine
> which may act not worse than current one on co-located queries, provide a
> boost for queries, using distributed joins, and provide an ability to execute
> arbitrary queries, requiring more than one map-reduce step in execution flow.
> [IEP
> link|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=130028084]
> [Dev list
> thread|http://apache-ignite-developers.2346864.n4.nabble.com/New-SQL-execution-engine-td43724.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-11907) Registration of continuous query should fail if nodes don't have remote filter class
[
https://issues.apache.org/jira/browse/IGNITE-11907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881229#comment-16881229
]
Roman Kondakov commented on IGNITE-11907:
-
[~Pavlukhin], patch looks good for me except two minors:
# Todo on {{IncompleteDeserializationException.java:22}}
# Misspelling in "registration" word {{StartRoutineDiscoveryMessage.java:34}}
> Registration of continuous query should fail if nodes don't have remote
> filter class
>
>
> Key: IGNITE-11907
> URL: https://issues.apache.org/jira/browse/IGNITE-11907
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.7
>Reporter: Denis Mekhanikov
>Assignee: Ivan Pavlukhin
>Priority: Major
> Attachments:
> ContinuousQueryRemoteFilterMissingInClassPathSelfTest.java
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> If one of data nodes doesn't have a remote filter class, then registration of
> continuous queries should fail with an exception. Currently nodes fail
> instead.
> Reproducer is attached:
> [^ContinuousQueryRemoteFilterMissingInClassPathSelfTest.java]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11948) Row count statistics tests fail frequently
[ https://issues.apache.org/jira/browse/IGNITE-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880609#comment-16880609 ] Roman Kondakov commented on IGNITE-11948: - [~Pavlukhin], I've fixed array casting: it looks like an old approach leads to the errors for java greater than 8. Please, take a look. Only the test code has been changed, so there is no visa from bot. The relevant suites looks good ([1|https://ci.ignite.apache.org/viewLog.html?buildId=4291323=queuedBuildOverviewTab], [2|https://ci.ignite.apache.org/viewLog.html?buildId=4291325=queuedBuildOverviewTab]). Patch is ready for review and merge. > Row count statistics tests fail frequently > -- > > Key: IGNITE-11948 > URL: https://issues.apache.org/jira/browse/IGNITE-11948 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 2.7.5 >Reporter: Ivan Pavlukhin >Assignee: Roman Kondakov >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > RowCountTableStatisticsSurvivesNodeRestartTest and > RowCountTableStatisticsUsageTest fail frequently on TC > https://ci.ignite.apache.org/viewLog.html?buildId=4238573=IgniteTests24Java8_BinaryObjectsSimpleMapperQueries -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IGNITE-11948) Row count statistics tests fail frequently
[ https://issues.apache.org/jira/browse/IGNITE-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov reassigned IGNITE-11948: --- Assignee: Roman Kondakov > Row count statistics tests fail frequently > -- > > Key: IGNITE-11948 > URL: https://issues.apache.org/jira/browse/IGNITE-11948 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 2.7.5 >Reporter: Ivan Pavlukhin >Assignee: Roman Kondakov >Priority: Major > > RowCountTableStatisticsSurvivesNodeRestartTest and > RowCountTableStatisticsUsageTest fail frequently on TC > https://ci.ignite.apache.org/viewLog.html?buildId=4238573=IgniteTests24Java8_BinaryObjectsSimpleMapperQueries -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-11256) Implement read-only mode for grid
[ https://issues.apache.org/jira/browse/IGNITE-11256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16854628#comment-16854628 ] Roman Kondakov commented on IGNITE-11256: - [~antonovsergey93], looks good for me. > Implement read-only mode for grid > - > > Key: IGNITE-11256 > URL: https://issues.apache.org/jira/browse/IGNITE-11256 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Sergey Antonov >Priority: Major > Fix For: 2.8 > > Time Spent: 3.5h > Remaining Estimate: 0h > > Should be triggered from control.sh utility. > Useful for maintenance work, for example checking partition consistency > (idle_verify) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-11256) Implement read-only mode for grid
[
https://issues.apache.org/jira/browse/IGNITE-11256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16852024#comment-16852024
]
Roman Kondakov commented on IGNITE-11256:
-
[~antonovsergey93], [~agoncharuk] in my opinion we need to keep
{{org.h2.api.ErrorCode}} and
{{org.apache.ignite.internal.processors.odbc.SqlStateCode}} in consistent state.
> Implement read-only mode for grid
> -
>
> Key: IGNITE-11256
> URL: https://issues.apache.org/jira/browse/IGNITE-11256
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexei Scherbakov
>Assignee: Sergey Antonov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Should be triggered from control.sh utility.
> Useful for maintenance work, for example checking partition consistency
> (idle_verify)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11756) SQL: implement a table row count statistics for the local queries
[
https://issues.apache.org/jira/browse/IGNITE-11756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847524#comment-16847524
]
Roman Kondakov commented on IGNITE-11756:
-
[~amashenkov], [~Pavlukhin], patch is ready for merge. Tests look good. Code
style is broken in the master branch.
> SQL: implement a table row count statistics for the local queries
> -
>
> Key: IGNITE-11756
> URL: https://issues.apache.org/jira/browse/IGNITE-11756
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Row count statistics should help the H2 optimizer to select the better query
> execution plan. Currently the row count supplied to H2 engine is hardcoded
> value == 1 (see {{org.h2.index.Index#getRowCountApproximation}}). As a
> first step we can provide an actual table size in the case of local query. To
> prevent counting size on each invocation we can cache row count value and
> invalidate it in some cases:
> * Rebalancing
> * Multiple updates (after the initial loading)
> * On timeout (i.e. 1 minute)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11756) SQL: implement a table row count statistics for the local queries
[
https://issues.apache.org/jira/browse/IGNITE-11756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845699#comment-16845699
]
Roman Kondakov commented on IGNITE-11756:
-
[~tledkov-gridgain], [~amashenkov], patch is ready for review. Tests look good.
> SQL: implement a table row count statistics for the local queries
> -
>
> Key: IGNITE-11756
> URL: https://issues.apache.org/jira/browse/IGNITE-11756
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Row count statistics should help the H2 optimizer to select the better query
> execution plan. Currently the row count supplied to H2 engine is hardcoded
> value == 1 (see {{org.h2.index.Index#getRowCountApproximation}}). As a
> first step we can provide an actual table size in the case of local query. To
> prevent counting size on each invocation we can cache row count value and
> invalidate it in some cases:
> * Rebalancing
> * Multiple updates (after the initial loading)
> * On timeout (i.e. 1 minute)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11756) SQL: implement a table row count statistics for the local queries
[
https://issues.apache.org/jira/browse/IGNITE-11756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16842136#comment-16842136
]
Roman Kondakov commented on IGNITE-11756:
-
[~tledkov-gridgain], [~amashenkov], patch is ready for review. Tests look good.
> SQL: implement a table row count statistics for the local queries
> -
>
> Key: IGNITE-11756
> URL: https://issues.apache.org/jira/browse/IGNITE-11756
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Row count statistics should help the H2 optimizer to select the better query
> execution plan. Currently the row count supplied to H2 engine is hardcoded
> value == 1 (see {{org.h2.index.Index#getRowCountApproximation}}). As a
> first step we can provide an actual table size in the case of local query. To
> prevent counting size on each invocation we can cache row count value and
> invalidate it in some cases:
> * Rebalancing
> * Multiple updates (after the initial loading)
> * On timeout (i.e. 1 minute)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11756) SQL: implement a table row count statistics for the local queries
[
https://issues.apache.org/jira/browse/IGNITE-11756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835627#comment-16835627
]
Roman Kondakov commented on IGNITE-11756:
-
[~amashenkov], [~tledkov-gridgain], [~taras.ledkov], patch is ready for review.
Tests look good.
> SQL: implement a table row count statistics for the local queries
> -
>
> Key: IGNITE-11756
> URL: https://issues.apache.org/jira/browse/IGNITE-11756
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Row count statistics should help the H2 optimizer to select the better query
> execution plan. Currently the row count supplied to H2 engine is hardcoded
> value == 1 (see {{org.h2.index.Index#getRowCountApproximation}}). As a
> first step we can provide an actual table size in the case of local query. To
> prevent counting size on each invocation we can cache row count value and
> invalidate it in some cases:
> * Rebalancing
> * Multiple updates (after the initial loading)
> * On timeout (i.e. 1 minute)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11756) SQL: implement a table row count statistics for the local queries
[
https://issues.apache.org/jira/browse/IGNITE-11756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835625#comment-16835625
]
Roman Kondakov commented on IGNITE-11756:
-
Local statistics was implemented by overriding method
{{GridH2IndexBase#getRowCountApproximation}} which is used by H2 optimizer in
order to estimate plan cost.
Row count statistics is cached to avoid recalculation on each method
invocation. This cache is invalidated on each table size changes.
> SQL: implement a table row count statistics for the local queries
> -
>
> Key: IGNITE-11756
> URL: https://issues.apache.org/jira/browse/IGNITE-11756
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Row count statistics should help the H2 optimizer to select the better query
> execution plan. Currently the row count supplied to H2 engine is hardcoded
> value == 1 (see {{org.h2.index.Index#getRowCountApproximation}}). As a
> first step we can provide an actual table size in the case of local query. To
> prevent counting size on each invocation we can cache row count value and
> invalidate it in some cases:
> * Rebalancing
> * Multiple updates (after the initial loading)
> * On timeout (i.e. 1 minute)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-11839) SQL: table join order changes may lead to incorrect result
[
https://issues.apache.org/jira/browse/IGNITE-11839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-11839:
Description:
Under some circumstances table join order changes may lead to incorrect result.
For example if one of joining tables is {{REPLICATED}} and another has
{{queryparallelism > 1}}.
This problem can be reproduced in test
{{IgniteSqlSegmentedIndexSelfTest#testSegmentedPartitionedWithReplicated}} if
swap tables {{Person}} and {{Organization}} in the method
{{IgniteSqlSegmentedIndexSelfTest#checkLocalQueryWithSegmentedIndex}} and set
{{enforceJoinOrder}} flag to {{true}}:
{code:java}
String select0 = "select o.name n1, p.name n2 from \"org\".Organization o,
\"pers\".Person p where p.orgId = o._key";
List> res = c1.query(new
SqlFieldsQuery(select0).setLocal(true).setEnforceJoinOrder(true)).getAll();
{code}
Result is:
{noformat}
java.lang.AssertionError:
Expected :956
Actual :8
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.ignite.testframework.junits.JUnitAssertAware.assertEquals(JUnitAssertAware.java:89)
at
org.apache.ignite.internal.processors.query.IgniteSqlSegmentedIndexSelfTest.checkLocalQueryWithSegmentedIndex(IgniteSqlSegmentedIndexSelfTest.java:280)
at
org.apache.ignite.internal.processors.query.IgniteSqlSegmentedIndexSelfTest.testSegmentedPartitionedWithReplicated(IgniteSqlSegmentedIndexSelfTest.java:222)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2145)
at java.lang.Thread.run(Thread.java:748)
{noformat}
was:
Under some circumstances table join order changes may lead to incorrect result.
For example if one of joining tables is {{REPLICATED}} and another has
{{queryparallelism > 1}}.
This problem can be reproduced in test
{{IgniteSqlSegmentedIndexSelfTest#testSegmentedPartitionedWithReplicated}} if
swap tables {{Person}} and {{Organization}} in the method
{{IgniteSqlSegmentedIndexSelfTest#checkLocalQueryWithSegmentedIndex}} and set
{{enforceJoinOrder}} flag to {{true}}:
{code:java}
String select0 = "select o.name n1, p.name n2 from \"org\".Organization o,
\"pers\".Person p where p.orgId = o._key";
List> res = c1.query(new
SqlFieldsQuery(select0).setLocal(true).setEnforceJoinOrder(true)).getAll();
{code}
> SQL: table join order changes may lead to incorrect result
> --
>
> Key: IGNITE-11839
> URL: https://issues.apache.org/jira/browse/IGNITE-11839
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.7
>Reporter: Roman Kondakov
>Priority: Major
> Fix For: 2.8
>
>
> Under some circumstances table join order changes may lead to incorrect
> result. For example if one of joining tables is {{REPLICATED}} and another
> has {{queryparallelism > 1}}.
> This problem can be reproduced in test
> {{IgniteSqlSegmentedIndexSelfTest#testSegmentedPartitionedWithReplicated}} if
> swap tables {{Person}} and {{Organization}} in the method
> {{IgniteSqlSegmentedIndexSelfTest#checkLocalQueryWithSegmentedIndex}} and set
> {{enforceJoinOrder}} flag to {{true}}:
> {code:java}
> String select0 = "select o.name n1, p.name n2 from \"org\".Organization o,
> \"pers\".Person p where p.orgId = o._key";
> List> res = c1.query(new
> SqlFieldsQuery(select0).setLocal(true).setEnforceJoinOrder(true)).getAll();
> {code}
> Result is:
> {noformat}
> java.lang.AssertionError:
> Expected :956
> Actual :8
>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
[jira] [Updated] (IGNITE-11839) SQL: table join order changes may lead to incorrect result
[
https://issues.apache.org/jira/browse/IGNITE-11839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-11839:
Description:
Under some circumstances table join order changes may lead to incorrect result.
For example if one of joining tables is {{REPLICATED}} and another has
{{queryparallelism > 1}}.
This problem can be reproduced in test
{{IgniteSqlSegmentedIndexSelfTest#testSegmentedPartitionedWithReplicated}} if
swap tables {{Person}} and {{Organization}} in the method
{{IgniteSqlSegmentedIndexSelfTest#checkLocalQueryWithSegmentedIndex}} and set
{{enforceJoinOrder}} flag to {{true}}:
{code:java}
String select0 = "select o.name n1, p.name n2 from \"org\".Organization o,
\"pers\".Person p where p.orgId = o._key";
List> res = c1.query(new
SqlFieldsQuery(select0).setLocal(true).setEnforceJoinOrder(true)).getAll();
{code}
was:
Under some circumstances table join order changes may lead to incorrect result.
For example if one of joining tables is {{REPLICATED}} and another has
{{queryparallelism > 1}}.
This problem can be reproduced in test
{{IgniteSqlSegmentedIndexSelfTest#testSegmentedPartitionedWithReplicated}} if
swap tables {{Person}} and {{Organization}} in the method
{{IgniteSqlSegmentedIndexSelfTest#checkLocalQueryWithSegmentedIndex}} and set
{{enforceJoinOrder}} flag to {{true}}.
> SQL: table join order changes may lead to incorrect result
> --
>
> Key: IGNITE-11839
> URL: https://issues.apache.org/jira/browse/IGNITE-11839
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.7
>Reporter: Roman Kondakov
>Priority: Major
> Fix For: 2.8
>
>
> Under some circumstances table join order changes may lead to incorrect
> result. For example if one of joining tables is {{REPLICATED}} and another
> has {{queryparallelism > 1}}.
> This problem can be reproduced in test
> {{IgniteSqlSegmentedIndexSelfTest#testSegmentedPartitionedWithReplicated}} if
> swap tables {{Person}} and {{Organization}} in the method
> {{IgniteSqlSegmentedIndexSelfTest#checkLocalQueryWithSegmentedIndex}} and set
> {{enforceJoinOrder}} flag to {{true}}:
> {code:java}
> String select0 = "select o.name n1, p.name n2 from \"org\".Organization o,
> \"pers\".Person p where p.orgId = o._key";
> List> res = c1.query(new
> SqlFieldsQuery(select0).setLocal(true).setEnforceJoinOrder(true)).getAll();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-11839) SQL: table join order changes may lead to incorrect result
Roman Kondakov created IGNITE-11839:
---
Summary: SQL: table join order changes may lead to incorrect result
Key: IGNITE-11839
URL: https://issues.apache.org/jira/browse/IGNITE-11839
Project: Ignite
Issue Type: Bug
Components: sql
Affects Versions: 2.7
Reporter: Roman Kondakov
Fix For: 2.8
Under some circumstances table join order changes may lead to incorrect result.
For example if one of joining tables is {{REPLICATED}} and another has
{{queryparallelism > 1}}.
This problem can be reproduced in test
{{IgniteSqlSegmentedIndexSelfTest#testSegmentedPartitionedWithReplicated}} if
swap tables {{Person}} and {{Organization}} in the method
{{IgniteSqlSegmentedIndexSelfTest#checkLocalQueryWithSegmentedIndex}} and set
{{enforceJoinOrder}} flag to {{true}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11251) SQL: getMoreResults() infinitely reports about update operation on zeroCursor
[
https://issues.apache.org/jira/browse/IGNITE-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821148#comment-16821148
]
Roman Kondakov commented on IGNITE-11251:
-
[~alapin], fix looks good for me. But anyway, IMO we should avoid using java
{{assert}} keyword for checking test results in the future.
> SQL: getMoreResults() infinitely reports about update operation on zeroCursor
> -
>
> Key: IGNITE-11251
> URL: https://issues.apache.org/jira/browse/IGNITE-11251
> Project: Ignite
> Issue Type: Bug
> Components: jdbc, sql
>Reporter: Pavel Kuznetsov
>Assignee: Alexander Lapin
>Priority: Major
> Labels: jdbc
> Fix For: 2.8
>
>
> If we got sql query that contain empty statement, jdbc thin driver will
> allways return {{true}} from {{getMoreResults}} method.
> Jdbc spec says:
> {noformat}
> oves to this Statement object's next result, returns true if it is a
> ResultSet object,
> <...>
> Returns:
> true if the next result is a ResultSet object; false if it is an update count
> or there are no more results
> {noformat}
> so test :
> {code:java}
> @Test
> public void test () throws Exception {
> try (Connection c = GridTestUtils.connect(grid(0), null)) {
> try (PreparedStatement p = c.prepareStatement(";")) {
> boolean isResultSet = p.execute();
> // Adding next line works the problem around:
> // p.getUpdateCount() == 0;
>
> boolean isResultSetReturned = p.getMoreResults();
>
>// should be false:
> assertFalse(isResultSetReturned); // == true
> }
> }
> }
> {code}
> {{getResultSet}} is {{null}} in this case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-11764) SQL: Do not use two step query in the case of single node cluster
Roman Kondakov created IGNITE-11764: --- Summary: SQL: Do not use two step query in the case of single node cluster Key: IGNITE-11764 URL: https://issues.apache.org/jira/browse/IGNITE-11764 Project: Ignite Issue Type: Improvement Components: sql Reporter: Roman Kondakov We can avoid using two step query in the case of the single-node cluster. The decision about the local execution should be made on the reducer step as well as partition pruning optimization does. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IGNITE-11756) SQL: implement a table row count statistics for the local queries
[
https://issues.apache.org/jira/browse/IGNITE-11756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov reassigned IGNITE-11756:
---
Assignee: Roman Kondakov
> SQL: implement a table row count statistics for the local queries
> -
>
> Key: IGNITE-11756
> URL: https://issues.apache.org/jira/browse/IGNITE-11756
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
>
> Row count statistics should help the H2 optimizer to select the better query
> execution plan. Currently the row count supplied to H2 engine is hardcoded
> value == 1 (see {{org.h2.index.Index#getRowCountApproximation}}). As a
> first step we can provide an actual table size in the case of local query. To
> prevent counting size on each invocation we can cache row count value and
> invalidate it in some cases:
> * Rebalancing
> * Multiple updates (after the initial loading)
> * On timeout (i.e. 1 minute)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-11756) SQL: implement a table row count statistics for the local queries
Roman Kondakov created IGNITE-11756:
---
Summary: SQL: implement a table row count statistics for the local
queries
Key: IGNITE-11756
URL: https://issues.apache.org/jira/browse/IGNITE-11756
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Roman Kondakov
Row count statistics should help the H2 optimizer to select the better query
execution plan. Currently the row count supplied to H2 engine is hardcoded
value == 1 (see {{org.h2.index.Index#getRowCountApproximation}}). As a
first step we can provide an actual table size in the case of local query. To
prevent counting size on each invocation we can cache row count value and
invalidate it in some cases:
* Rebalancing
* Multiple updates (after the initial loading)
* On timeout (i.e. 1 minute)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-11712) SQL: review security check for SQL/DML queries
Roman Kondakov created IGNITE-11712: --- Summary: SQL: review security check for SQL/DML queries Key: IGNITE-11712 URL: https://issues.apache.org/jira/browse/IGNITE-11712 Project: Ignite Issue Type: Improvement Components: sql Reporter: Roman Kondakov Currently the security check (read/write permissions) is carried out during the query execution. It involves some extra actions (like a query registration) which can be avoided if the security check is conducted on the earlier stage of the query execution, for example right away after the parsing. For SELECT queries only read permission should be checked. For INSERT queries without SELECT only write permission should be checked. For UPDATE queries or INSERT queries with SELECT both read and write permissions should be checked. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-11673) SQL: It looks like security check is missed in h2 indexing.
[
https://issues.apache.org/jira/browse/IGNITE-11673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809685#comment-16809685
]
Roman Kondakov commented on IGNITE-11673:
-
[~amashenkov], [~vozerov], I've removed an extra security check from
{{executeSelect}}. Tests look good. Patch is ready for review.
> SQL: It looks like security check is missed in h2 indexing.
> ---
>
> Key: IGNITE-11673
> URL: https://issues.apache.org/jira/browse/IGNITE-11673
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Security check is no longer conducted in {{IgniteH2Indexing#executeSelect0}}
> after IGNITE-10104 having been merged.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11679) MVCC: Security check is violated for MVCC caches
[ https://issues.apache.org/jira/browse/IGNITE-11679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808712#comment-16808712 ] Roman Kondakov commented on IGNITE-11679: - See corresponding IGN ticket for details > MVCC: Security check is violated for MVCC caches > > > Key: IGNITE-11679 > URL: https://issues.apache.org/jira/browse/IGNITE-11679 > Project: Ignite > Issue Type: Bug > Components: mvcc >Reporter: Roman Kondakov >Priority: Major > > Security check is not applied for MVCC caches. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11679) MVCC: Security check is violated for MVCC caches
Roman Kondakov created IGNITE-11679: --- Summary: MVCC: Security check is violated for MVCC caches Key: IGNITE-11679 URL: https://issues.apache.org/jira/browse/IGNITE-11679 Project: Ignite Issue Type: Bug Components: mvcc Reporter: Roman Kondakov Security check is not applied for MVCC caches. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-11673) SQL: It looks like security check is missed in h2 indexing.
[
https://issues.apache.org/jira/browse/IGNITE-11673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808654#comment-16808654
]
Roman Kondakov commented on IGNITE-11673:
-
[~amashenkov] it seems that DML and Commands permissions are checked in another
places. For example, DML is checked in {{GridNearTxLocal#beforePut}} or
{{GridDhtAtomicCache#updateAll0}} or several another places. Commands are
checked in {{CommandProcessor#runCommandH2}}.
But it looks like something goes wrong in MVCC case. I'll create a ticket.
> SQL: It looks like security check is missed in h2 indexing.
> ---
>
> Key: IGNITE-11673
> URL: https://issues.apache.org/jira/browse/IGNITE-11673
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Security check is no longer conducted in {{IgniteH2Indexing#executeSelect0}}
> after IGNITE-10104 having been merged.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11673) SQL: It looks like security check is missed in h2 indexing.
[
https://issues.apache.org/jira/browse/IGNITE-11673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808474#comment-16808474
]
Roman Kondakov commented on IGNITE-11673:
-
[~vozerov], [~amashenkov], please, review the patch. Tests look good: there are
some problems with PHP and Python clients in the master branch.
> SQL: It looks like security check is missed in h2 indexing.
> ---
>
> Key: IGNITE-11673
> URL: https://issues.apache.org/jira/browse/IGNITE-11673
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Security check is no longer conducted in {{IgniteH2Indexing#executeSelect0}}
> after IGNITE-10104 having been merged.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-11673) SQL: It looks like security check is missed in h2 indexing.
Roman Kondakov created IGNITE-11673:
---
Summary: SQL: It looks like security check is missed in h2
indexing.
Key: IGNITE-11673
URL: https://issues.apache.org/jira/browse/IGNITE-11673
Project: Ignite
Issue Type: Bug
Components: sql
Reporter: Roman Kondakov
Assignee: Roman Kondakov
Security check is no longer conducted in {{IgniteH2Indexing#executeSelect0}}
after IGNITE-10104 having been merged.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
