Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21737#discussion_r201758572
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -738,6 +738,10 @@ class Analyzer(
if
findAliases(aggregateExpressions).intersect(conflictingAttributes).nonEmpty =>
(oldVersion, oldVersion.copy(aggregateExpressions =
newAliases(aggregateExpressions)))
+ case oldVersion @ FlatMapGroupsInPandas(_, _, output, _)
+ if
AttributeSet(output).intersect(conflictingAttributes).nonEmpty =>
--- End diff --
We need to ensure all the expressions have unique IDs, instead of
deduplicating it when we hit conflicts.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
