Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/15763
@hvanhovell thank you for the trick.
Exploring your idea further, I looked at CheckAnalysis. This is a trait
that Analyzer extends from. The checks in def CheckAnalysis of trait
CheckAnalysis walk the operators in a LogicalPlan tree and look for violations
specified to each operator type. The checks in Analyzer/ResolveSubquery has the
context of a subquery, specifically the connect of those correlated predicates
in the subquery, and look at everything from the subquery's viewpoint to see if
there exists any violation then report an exception. I guess finding a
correlated subquery for each operator type and flagging a violation is more
expensive than here we having a correlated subquery, making sure no violation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
