ritikam2 commented on code in PR #40116:
URL: https://github.com/apache/spark/pull/40116#discussion_r1141630253
##########
sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala:
##########
@@ -89,9 +89,22 @@ class RelationalGroupedDataset protected[sql](
case expr: NamedExpression => expr
case a: AggregateExpression if
a.aggregateFunction.isInstanceOf[TypedAggregateExpression] =>
UnresolvedAlias(a, Some(Column.generateAlias))
- case expr: Expression => Alias(expr, toPrettySQL(expr))()
+ case ag: UnresolvedFunction if (containsStar(Seq(ag))) || ag.isDistinct =>
+ UnresolvedAlias(expr, None)
+ case expr: Expression =>
+ if (containsStar(Seq(expr))) {
+ UnresolvedAlias(expr, None)
+ } else {
+ Alias(expr, toPrettySQL(expr))()
Review Comment:
> If we want a surgical fix, shall we fix how `toPrettySQL` handles star?
How can we fix toPrettySQL to provide the unresolved values of an expr ? The
best we can do is to make it count(distinct *) which is still not the same as
what spark.sql produces.
Besides it seems like that the alias in case of aggreagate values in
dataframe is getting ignored.
scala> df.groupBy("id").agg(count_distinct($"*")).alias("count_alias").show()
+---+-----------------------+
| id|count(unresolvedstar())|
+---+-----------------------+
| 1| 1|
| 2| 1|
| 3| 1|
| 4| 1|
| 5| 1|
| 6| 1|
| 7| 1|
| 8| 1|
| 9| 1|
+---+-----------------------+
scala> spark.sql(" SELECT id, COUNT(DISTINCT *) as count_alias FROM table
GROUP BY id ").show()
+---+-----------+
| id|count_alias|
+---+-----------+
| 1| 1|
| 2| 1|
| 3| 1|
| 4| 1|
| 5| 1|
| 6| 1|
| 7| 1|
| 8| 1|
| 9| 1|
+---+-----------+
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
