[
https://issues.apache.org/jira/browse/CALCITE-7574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18085169#comment-18085169
]
leishp commented on CALCITE-7574:
---------------------------------
Hi [~jensen], thanks for the review.
I've simplified the test case to a minimal form. The nesting is unfortunately
necessary to reproduce the issue:
- The outer FROM must be an Aggregate-derived table, so that the correlated
condition references an aggregate result field (t.s = SUM(sal)) rather than a
group field.
- The subquery must contain COUNT(*) to create a second Aggregate node in the
decorrelation plan.
- The correlation condition e.sal > t.s must reference this aggregate result
field.
Removing any of these three conditions makes the bug unreproducible. The
minimal SQL is:
{code:sql}
SELECT t.deptno,
(SELECT COUNT(*) FROM emp e
WHERE e.deptno = t.deptno AND e.sal > t.s)
FROM (SELECT deptno, SUM(sal) AS s FROM emp GROUP BY deptno) t
{code}
I've updated the PR with this minimal test case and also updated the JIRA
description accordingly.
> RelDecorrelator.isFieldNotNullRecursive throws IndexOutOfBoundsException when
> decorrelating correlated scalar subquery with Aggregate
> -------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CALCITE-7574
> URL: https://issues.apache.org/jira/browse/CALCITE-7574
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.41.0, 1.42.0
> Reporter: leishp
> Assignee: leishp
> Priority: Major
> Labels: pull-request-available
>
> `RelDecorrelator.isFieldNotNullRecursive` throws `IndexOutOfBoundsException`
> when decorrelating a correlated scalar subquery that involves an Aggregate.
> The bug was introduced in CALCITE-6962 which added the
> `isFieldNotNull`/`isFieldNotNullRecursive` methods.
> h3. Root Cause
> In `RelDecorrelator.isFieldNotNullRecursive`, the Aggregate branch at line
> 3856:
>
> {code:java}
> } else if (rel instanceof Aggregate) {
> Aggregate agg = (Aggregate) rel;
> ImmutableBitSet groupSet = agg.getGroupSet();
> if (index >= groupSet.size())
> { // BUG: should be agg.getGroupCount() return false; }
> return isFieldNotNullRecursive(agg.getInput(),
> groupSet.asList().get(index));
> }
> {code}
>
> - `ImmutableBitSet.size()` returns the bitset {*}{{*}}capacity{{*}}{*}
> (`words.length * 64`), typically 64 or more
> - `groupSet.asList().get(index)` internally calls `nth(index)`, which
> requires `index < cardinality()`
> - `Aggregate.getGroupCount()` returns `groupSet.cardinality()`, the actual
> number of group keys
> When a field index corresponds to an aggregate result field (not a group
> field), the bounds check `index >= groupSet.size()` is too lenient (e.g., `1
> >= 64` is false), allowing execution to proceed to `nth(1)` which throws
> `IndexOutOfBoundsException` because the bitset has only 1 set bit.
> h3. h3. Reproduction SQL
> {code:sql}
> select t.deptno,
> (select count(*) from emp e
> where e.deptno = t.deptno and e.sal > t.s
> )
> from (select deptno, sum(sal) as s
> from emp group by deptno
> ) t{code}
> h3. h3. Exception
> {code:java}
> java.lang.IndexOutOfBoundsException: index out of range: 1
> at org.apache.calcite.util.ImmutableBitSet.nth(ImmutableBitSet.java:903)
> at org.apache.calcite.util.ImmutableBitSet$2.get(ImmutableBitSet.java:681)
> at org.apache.calcite.util.ImmutableBitSet$2.get(ImmutableBitSet.java:679)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNullRecursive(RelDecorrelator.java:3859)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNullRecursive(RelDecorrelator.java:3851)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNullRecursive(RelDecorrelator.java:3859)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNullRecursive(RelDecorrelator.java:3872)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNullRecursive(RelDecorrelator.java:3851)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNull(RelDecorrelator.java:3840)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1902)
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
