leishp created CALCITE-7574:
-------------------------------
Summary: RelDecorrelator.isFieldNotNullRecursive throws
IndexOutOfBoundsException when decorrelating correlated scalar subquery with
Aggregate
Key: CALCITE-7574
URL: https://issues.apache.org/jira/browse/CALCITE-7574
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.42.0, 1.41.0
Reporter: leishp
`RelDecorrelator.isFieldNotNullRecursive` throws `IndexOutOfBoundsException`
when decorrelating a correlated scalar subquery that involves an Aggregate. The
bug was introduced in
[[CALCITE-6962]|https://issues.apache.org/jira/browse/CALCITE-6962] which added
the `isFieldNotNull`/`isFieldNotNullRecursive` methods.
### Root Cause
In `RelDecorrelator.isFieldNotNullRecursive`, the Aggregate branch at line 3856:
```java
} else if (rel instanceof Aggregate) {
Aggregate agg = (Aggregate) rel;
ImmutableBitSet groupSet = agg.getGroupSet();
if (index >= groupSet.size()) { // BUG: should be agg.getGroupCount()
return false;
}
return isFieldNotNullRecursive(agg.getInput(),
groupSet.asList().get(index));
}
```
- `ImmutableBitSet.size()` returns the bitset **capacity** (`words.length *
64`), typically 64 or more
- `groupSet.asList().get(index)` internally calls `nth(index)`, which requires
`index < cardinality()`
- `Aggregate.getGroupCount()` returns `groupSet.cardinality()`, the actual
number of group keys
When a field index corresponds to an aggregate result field (not a group
field), the bounds check `index >= groupSet.size()` is too lenient (e.g., `1 >=
64` is false), allowing execution to proceed to `nth(1)` which throws
`IndexOutOfBoundsException` because the bitset has only 1 set bit.
### Reproduction SQL
```sql
select t1.deptno, t1.total_sal,
(select count(distinct t2.total_sal) + 1
from (select deptno, sum(sal) as total_sal
from emp where deptno is not null
group by deptno) t2
where t2.deptno = t1.deptno
and t2.total_sal > t1.total_sal) as rank_sal
from (select deptno, sum(sal) as total_sal
from emp where deptno is not null
group by deptno) t1
order by t1.deptno, t1.total_sal desc
```
### Exception
```
java.lang.IndexOutOfBoundsException: index out of range: 1
at org.apache.calcite.util.ImmutableBitSet.nth(ImmutableBitSet.java:903)
at org.apache.calcite.util.ImmutableBitSet$2.get(ImmutableBitSet.java:681)
at org.apache.calcite.util.ImmutableBitSet$2.get(ImmutableBitSet.java:679)
at
org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNullRecursive(RelDecorrelator.java:3859)
at
org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNullRecursive(RelDecorrelator.java:3851)
at
org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNullRecursive(RelDecorrelator.java:3859)
at
org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNullRecursive(RelDecorrelator.java:3872)
at
org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNullRecursive(RelDecorrelator.java:3851)
at
org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNull(RelDecorrelator.java:3840)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1902)
...
```
##
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
