HyukjinKwon commented on code in PR #59293:
URL: https://github.com/apache/spark/pull/59293#discussion_r4236318288
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala:
##########
@@ -1782,6 +1782,10 @@ class Analyzer(
}
expanded
}
+ // If the pivot aggregate function argument contains Stars, expand it,
e.g. count(*) to
+ // count(1), as for Aggregate.
+ case p: Pivot if containsStar(p.aggregates) =>
Review Comment:
**Nit (P3):** This arm only makes progress when the star is an argument of
another expression. A bare top-level `*` in the pivot aggregate list comes back
from `expandStarExpression` unchanged. Examples are `SELECT * FROM t PIVOT
(sum(v), * FOR y IN (1, 2))`, or a DataFrame/Connect pivot whose grouping or
pivot column is still unresolved, with `.agg(col("*"))`.
`containsStar(p.aggregates)` then stays true and this case keeps matching on
every iteration, so the Pivot never reaches the catch-all `case q: LogicalPlan
=> q.mapExpressions(resolveExpressionByPlanChildren(...))` that resolves its
pivot column and grouping expressions. `CheckAnalysis` walks
`Pivot.expressions` in field order and fails first with `UNRESOLVED_COLUMN` on
`y`, a column that exists. Before this change, these queries got
`INVALID_USAGE_OF_STAR_OR_REGEX ... in Pivot`.
The `Aggregate` arm avoids this because `buildExpandedProjectList` handles a
top-level `Star` explicitly. Here it may be simplest to reject a top-level star
up front with `QueryCompilationErrors.invalidStarUsageError("Pivot", ...)`, as
the `Generate` arm does, and run `expandStarExpression` only for nested stars,
plus a negative test.
**Verification:**
- **Regression:** An analyzer-level negative test with a top-level star in
Pivot aggregates, alongside a pivot column or grouping expression that is still
unresolved, asserts INVALID_USAGE_OF_STAR_OR_REGEX with Pivot as the operator.
- **Behavior:** The existing count(*) pivot analysis and DataFrame result
tests continue to pass unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
