maropu commented on a change in pull request #29585:
URL: https://github.com/apache/spark/pull/29585#discussion_r490341173
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala
##########
@@ -544,7 +548,7 @@ object RewriteCorrelatedScalarSubquery extends
Rule[LogicalPlan] {
Alias(
If(IsNull(alwaysTrueRef),
resultWithZeroTups.get,
- aggValRef), origOutput.name)(exprId = origOutput.exprId),
+ aggValRef), origOutput.name)(exprId = newExprId),
Review comment:
We might be able to do so, but we need more logic so that a parent node,
`Aggregate` and `Project`, can refer to an attribute with a newly-generated
expr ID (This topic is related to the comment:
https://github.com/apache/spark/pull/29585/files#r490216289).
In the current master, the parent node just refer to an attribute with the
expr Id of the original output (`origOutput.exprId`) because
`origOutput.exprId` is reused in this `Project`. If we generate a new expr Id
here, we need to rewrite the parent node later by using the generated expr ID.
If necessary, I will try this approach tomorrow, so please let me know.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
