This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 9d4cb165de56 [SPARK-58252][SQL] Throw GROUP_BY_POS_OUT_OF_RANGE for
out-of-range ordinal in pipe AGGREGATE GROUP BY
9d4cb165de56 is described below
commit 9d4cb165de5683c00ad5d6ff8243151775eedcf8
Author: YangJie <[email protected]>
AuthorDate: Wed Jul 22 12:39:30 2026 -0700
[SPARK-58252][SQL] Throw GROUP_BY_POS_OUT_OF_RANGE for out-of-range ordinal
in pipe AGGREGATE GROUP BY
### What changes were proposed in this pull request?
An out-of-range GROUP BY ordinal in the SQL pipe AGGREGATE operator threw a
raw `java.lang.IndexOutOfBoundsException` instead of a proper
`AnalysisException`. For example:
```sql
select 3 as x, 4 as y
|> aggregate sum(y) group by 5;
```
`resolvePipeAggregateExpressionOrdinal` resolved the ordinal with an
unguarded `inputs(index - 1)`, so an out-of-range position (or `0`) surfaced
the internal JVM exception.
This PR adds the same bounds guard the regular GROUP BY ordinal resolver
(`resolveGroupByExpressionOrdinal`) already uses, throwing
`GROUP_BY_POS_OUT_OF_RANGE` (sqlState `42805`). The
`UnresolvedPipeAggregateOrdinal` node is now created with the ordinal literal's
origin (via `CurrentOrigin.withOrigin`), so the error's `queryContext` points
at the offending ordinal rather than the whole pipe statement, matching regular
GROUP BY.
### Why are the changes needed?
A raw `IndexOutOfBoundsException` escaping the analyzer for valid-syntax
SQL is an internal error leak. User-facing errors should carry an error class,
sqlState, and query context. The pipe AGGREGATE path was the only pipe operator
with its own GROUP BY ordinal handling and the only one missing this guard;
regular `GROUP BY <ordinal>` has thrown `GROUP_BY_POS_OUT_OF_RANGE` with
precise context for years.
### Does this PR introduce _any_ user-facing change?
Yes, on the error path only. For an out-of-range or non-positive ordinal in
pipe AGGREGATE GROUP BY:
Before:
```
java.lang.IndexOutOfBoundsException: Index 4 out of bounds for length 2
```
After:
```
[GROUP_BY_POS_OUT_OF_RANGE] GROUP BY position 5 is not in select list
(valid range is [1, 2]). SQLSTATE: 42805
```
with a query context pointing at the ordinal. Valid (in-range) ordinals are
unaffected.
### How was this patch tested?
Added positive-range-adjacent negative cases to `pipe-operators.sql`
(`group by 5` above range, `group by 0` non-positive) in the aggregation
negative-tests section; regenerated the golden files. The regenerated goldens
assert the error class, sqlState, `index`/`size` parameters, and the precise
`queryContext` (fragment pointing at the ordinal). Existing positive
pipe-ordinal cases (including the exact upper bound `group by x, 2, 3`)
regenerated with no diff, confirming zero happy-pat [...]
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
Closes #57423 from LuciferYang/pipe-groupby-ordinal-oob.
Authored-by: YangJie <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 4462cb1dd30ac46ab27bef394c856be1e760757f)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../spark/sql/catalyst/analysis/Analyzer.scala | 13 ++++--
.../spark/sql/catalyst/parser/AstBuilder.scala | 7 +++-
.../analyzer-results/pipe-operators.sql.out | 44 ++++++++++++++++++++
.../resources/sql-tests/inputs/pipe-operators.sql | 8 ++++
.../sql-tests/results/pipe-operators.sql.out | 48 ++++++++++++++++++++++
5 files changed, 115 insertions(+), 5 deletions(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
index c629de82c6ff..215c859154a0 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
@@ -2252,11 +2252,18 @@ class Analyzer(
private def resolvePipeAggregateExpressionOrdinal(
expr: NamedExpression,
inputs: Seq[Attribute]): NamedExpression = expr match {
- case UnresolvedPipeAggregateOrdinal(index) =>
+ case ordinal @ UnresolvedPipeAggregateOrdinal(index) =>
// In this case, the user applied the SQL pipe aggregate operator ("|>
AGGREGATE") and used
// ordinals in its GROUP BY clause. This expression then refers to the
i-th attribute of the
- // child operator (one-based). Here we resolve the ordinal to the
corresponding attribute.
- inputs(index - 1)
+ // child operator (one-based). Here we resolve the ordinal to the
corresponding attribute, or
+ // throw GROUP_BY_POS_OUT_OF_RANGE if it is outside the range of the
child's attributes.
+ withPosition(ordinal) {
+ if (index > 0 && index <= inputs.size) {
+ inputs(index - 1)
+ } else {
+ throw QueryCompilationErrors.groupByPositionRangeError(index,
inputs.size)
+ }
+ }
case other =>
other
}
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
index 6b60cb9ebd3a..b18187da474a 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
@@ -7919,9 +7919,12 @@ class AstBuilder extends DataTypeAstBuilder
// grouping and aggregate expressions, respectively. This will let
the
// [[ResolveOrdinalInOrderByAndGroupBy]] rule detect the ordinal
in the aggregate list
// and replace it with the corresponding attribute from the child
operator.
- case UnresolvedOrdinal(v: Int) =>
+ case ordinal @ UnresolvedOrdinal(v: Int) =>
newGroupingExpressions +=
UnresolvedOrdinal(newAggregateExpressions.length + 1)
- newAggregateExpressions += UnresolvedPipeAggregateOrdinal(v)
+ // Preserve the ordinal's origin so an out-of-range error points
at the position
+ // itself, not the whole pipe statement (matching regular GROUP
BY).
+ newAggregateExpressions +=
+ CurrentOrigin.withOrigin(ordinal.origin) {
UnresolvedPipeAggregateOrdinal(v) }
case e: Expression =>
newGroupingExpressions += e
newAggregateExpressions += UnresolvedAlias(e, None)
diff --git
a/sql/core/src/test/resources/sql-tests/analyzer-results/pipe-operators.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/pipe-operators.sql.out
index d136341fa32b..666c5c91f8bf 100644
---
a/sql/core/src/test/resources/sql-tests/analyzer-results/pipe-operators.sql.out
+++
b/sql/core/src/test/resources/sql-tests/analyzer-results/pipe-operators.sql.out
@@ -3492,6 +3492,50 @@ org.apache.spark.sql.catalyst.parser.ParseException
}
+-- !query
+select 3 as x, 4 as y
+|> aggregate sum(y) group by 5
+-- !query analysis
+org.apache.spark.sql.AnalysisException
+{
+ "errorClass" : "GROUP_BY_POS_OUT_OF_RANGE",
+ "sqlState" : "42805",
+ "messageParameters" : {
+ "index" : "5",
+ "size" : "2"
+ },
+ "queryContext" : [ {
+ "objectType" : "",
+ "objectName" : "",
+ "startIndex" : 52,
+ "stopIndex" : 52,
+ "fragment" : "5"
+ } ]
+}
+
+
+-- !query
+select 3 as x, 4 as y
+|> aggregate sum(y) group by 0
+-- !query analysis
+org.apache.spark.sql.AnalysisException
+{
+ "errorClass" : "GROUP_BY_POS_OUT_OF_RANGE",
+ "sqlState" : "42805",
+ "messageParameters" : {
+ "index" : "0",
+ "size" : "2"
+ },
+ "queryContext" : [ {
+ "objectType" : "",
+ "objectName" : "",
+ "startIndex" : 52,
+ "stopIndex" : 52,
+ "fragment" : "0"
+ } ]
+}
+
+
-- !query
table courseSales
|> aggregate sum(earnings) group by rollup(course, `year`)
diff --git a/sql/core/src/test/resources/sql-tests/inputs/pipe-operators.sql
b/sql/core/src/test/resources/sql-tests/inputs/pipe-operators.sql
index 7347ad1385a3..916b114f17f5 100644
--- a/sql/core/src/test/resources/sql-tests/inputs/pipe-operators.sql
+++ b/sql/core/src/test/resources/sql-tests/inputs/pipe-operators.sql
@@ -1128,6 +1128,14 @@ table other
select 3 as x, 4 as y
|> aggregate group by all;
+-- The AGGREGATE GROUP BY ordinal position is out of range.
+select 3 as x, 4 as y
+|> aggregate sum(y) group by 5;
+
+-- The AGGREGATE GROUP BY ordinal position is not positive.
+select 3 as x, 4 as y
+|> aggregate sum(y) group by 0;
+
-- GROUP BY ROLLUP is not supported yet.
table courseSales
|> aggregate sum(earnings) group by rollup(course, `year`)
diff --git
a/sql/core/src/test/resources/sql-tests/results/pipe-operators.sql.out
b/sql/core/src/test/resources/sql-tests/results/pipe-operators.sql.out
index fabe74b49158..578e1478ea23 100644
--- a/sql/core/src/test/resources/sql-tests/results/pipe-operators.sql.out
+++ b/sql/core/src/test/resources/sql-tests/results/pipe-operators.sql.out
@@ -3109,6 +3109,54 @@ org.apache.spark.sql.catalyst.parser.ParseException
}
+-- !query
+select 3 as x, 4 as y
+|> aggregate sum(y) group by 5
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.sql.AnalysisException
+{
+ "errorClass" : "GROUP_BY_POS_OUT_OF_RANGE",
+ "sqlState" : "42805",
+ "messageParameters" : {
+ "index" : "5",
+ "size" : "2"
+ },
+ "queryContext" : [ {
+ "objectType" : "",
+ "objectName" : "",
+ "startIndex" : 52,
+ "stopIndex" : 52,
+ "fragment" : "5"
+ } ]
+}
+
+
+-- !query
+select 3 as x, 4 as y
+|> aggregate sum(y) group by 0
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.sql.AnalysisException
+{
+ "errorClass" : "GROUP_BY_POS_OUT_OF_RANGE",
+ "sqlState" : "42805",
+ "messageParameters" : {
+ "index" : "0",
+ "size" : "2"
+ },
+ "queryContext" : [ {
+ "objectType" : "",
+ "objectName" : "",
+ "startIndex" : 52,
+ "stopIndex" : 52,
+ "fragment" : "0"
+ } ]
+}
+
+
-- !query
table courseSales
|> aggregate sum(earnings) group by rollup(course, `year`)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
