[
https://issues.apache.org/jira/browse/IGNITE-27822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladislav Pyatkov updated IGNITE-27822:
---------------------------------------
Description:
h3. Motivation
Apache Calcite can parse, validate, and produce a logical plan for recursive
CTEs using LogicalRepeatUnion, transient tables, and table spools. However,
Ignite Calcite currently lacks the physical relational operators, planner
rules, and runtime execution nodes required to execute such plans.
As a result, valid hierarchical queries using {{WITH RECURSIVE}} cannot be
executed by the Ignite Calcite engine.
This task adds the first complete vertical slice of recursive CTE support.
h3. Definition of done
# Supported {{WITH RECURSIVE ... UNION ALL}} queries execute through the Ignite
Calcite engine.
# Recursive state is isolated between separate CTE instances and separate
queries.
# Recursive delta memory is included in SQL query memory accounting.
# Distributed source-table scans can feed coordinator-side recursion.
was:
h3. Motivation
SQL using "WITH RECURSIVE" is an ANSI standard. Without this type of query, it
is impossible to build a tree in the hierarchy table.
There is a simple example:
{noformat}
WITH RECURSIVE t(n) AS (SELECT 1 UNION ALL SELECT n + 1 FROM t WHERE n < 100)
SELECT sum(n) FROM t;
{noformat}
It currently returns an error.
{noformat}
java.lang.NullPointerException
at
com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:265)
at
org.apache.ignite.internal.processors.query.calcite.metadata.IgniteMdCollation.collations(IgniteMdCollation.java:167)
at
org.apache.calcite.rel.metadata.janino.GeneratedMetadata_CollationHandler.collations_$(Unknown
Source)
at
org.apache.calcite.rel.metadata.janino.GeneratedMetadata_CollationHandler.collations(Unknown
Source)
at
org.apache.calcite.rel.metadata.RelMetadataQuery.collations(RelMetadataQuery.java:628)
at
org.apache.calcite.rel.metadata.RelMdCollation.filter(RelMdCollation.java:277)
at
org.apache.calcite.rel.logical.LogicalFilter.lambda$create$0(LogicalFilter.java:138)
at org.apache.calcite.plan.RelTraitSet.replaceIfs(RelTraitSet.java:246)
at
org.apache.calcite.rel.logical.LogicalFilter.create(LogicalFilter.java:137)
at
org.apache.calcite.rel.core.RelFactories$FilterFactoryImpl.createFilter(RelFactories.java:375)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertWhere(SqlToRelConverter.java:1141)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:794)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:735)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3939)
at
org.apache.ignite.internal.processors.query.calcite.prepare.IgniteSqlToRelConvertor.convertQueryRecursive(IgniteSqlToRelConvertor.java:87)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertSetOp(SqlToRelConverter.java:4000)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3951)
at
org.apache.ignite.internal.processors.query.calcite.prepare.IgniteSqlToRelConvertor.convertQueryRecursive(IgniteSqlToRelConvertor.java:87)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2487)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2359)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2413)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2359)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2855)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2457)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2397)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2359)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:755)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:735)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3939)
at
org.apache.ignite.internal.processors.query.calcite.prepare.IgniteSqlToRelConvertor.convertQueryRecursive(IgniteSqlToRelConvertor.java:87)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:628)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertWith(SqlToRelConverter.java:4912)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3953)
at
org.apache.ignite.internal.processors.query.calcite.prepare.IgniteSqlToRelConvertor.convertQueryRecursive(IgniteSqlToRelConvertor.java:87)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:628)
at
org.apache.ignite.internal.processors.query.calcite.prepare.IgnitePlanner.rel(IgnitePlanner.java:348)
at
org.apache.ignite.internal.processors.query.calcite.prepare.PlannerHelper.optimize(PlannerHelper.java:109)
at
org.apache.ignite.internal.processors.query.calcite.prepare.PrepareServiceImpl.prepareQuery(PrepareServiceImpl.java:181)
at
org.apache.ignite.internal.processors.query.calcite.prepare.PrepareServiceImpl.prepareSingle(PrepareServiceImpl.java:107)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.lambda$parseAndProcessQuery$5(CalciteQueryProcessor.java:586)
at
org.apache.ignite.internal.processors.query.calcite.prepare.QueryPlanCacheImpl.lambda$queryPlan$0(QueryPlanCacheImpl.java:64)
at
java.base/java.util.concurrent.ConcurrentMap.computeIfAbsent(ConcurrentMap.java:330)
at
org.apache.ignite.internal.processors.query.calcite.prepare.QueryPlanCacheImpl.queryPlan(QueryPlanCacheImpl.java:64)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.lambda$parseAndProcessQuery$6(CalciteQueryProcessor.java:583)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.processQuery(CalciteQueryProcessor.java:779)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.parseAndProcessQuery(CalciteQueryProcessor.java:580)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.query(CalciteQueryProcessor.java:447)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor$2.applyx(GridQueryProcessor.java:3125)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor$2.applyx(GridQueryProcessor.java:3081)
at
org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:3867)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.lambda$querySqlFields$3(GridQueryProcessor.java:3158)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuerySafe(GridQueryProcessor.java:3295)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:3077)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:3001)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:801)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:749)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:412)
at
org.apache.ignite.internal.processors.cache.RecursiveCteQuery.queryAndPrint(RecursiveCteQuery.java:197)
at
org.apache.ignite.internal.processors.cache.RecursiveCteQuery.testSimpleExample(RecursiveCteQuery.java:110)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2486)
at java.base/java.lang.Thread.run(Thread.java:829)
{noformat}
h3. Definition of done
The query above shuld return 5050
Recursive query support
> Support recursive query for Calcite engine
> ------------------------------------------
>
> Key: IGNITE-27822
> URL: https://issues.apache.org/jira/browse/IGNITE-27822
> Project: Ignite
> Issue Type: Improvement
> Reporter: Vladislav Pyatkov
> Assignee: Vladislav Pyatkov
> Priority: Major
> Labels: ignite-2
> Attachments: RecursiveQueryTest.java
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h3. Motivation
> Apache Calcite can parse, validate, and produce a logical plan for recursive
> CTEs using LogicalRepeatUnion, transient tables, and table spools. However,
> Ignite Calcite currently lacks the physical relational operators, planner
> rules, and runtime execution nodes required to execute such plans.
> As a result, valid hierarchical queries using {{WITH RECURSIVE}} cannot be
> executed by the Ignite Calcite engine.
> This task adds the first complete vertical slice of recursive CTE support.
> h3. Definition of done
> # Supported {{WITH RECURSIVE ... UNION ALL}} queries execute through the
> Ignite Calcite engine.
> # Recursive state is isolated between separate CTE instances and separate
> queries.
> # Recursive delta memory is included in SQL query memory accounting.
> # Distributed source-table scans can feed coordinator-side recursion.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
