Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/13867#discussion_r75892139
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -912,19 +912,24 @@ class Analyzer(
* Resolve the correlated expressions in a subquery by using the an
outer plans' references. All
* resolved outer references are wrapped in an [[OuterReference]]
*/
- private def resolveOuterReferences(plan: LogicalPlan, outer:
LogicalPlan): LogicalPlan = {
+ private def resolveOuterReferences(
+ plan: LogicalPlan,
+ outers: Seq[LogicalPlan]): LogicalPlan = {
plan transformDown {
case q: LogicalPlan if q.childrenResolved && !q.resolved =>
q transformExpressions {
case u @ UnresolvedAttribute(nameParts) =>
withPosition(u) {
- try {
- outer.resolve(nameParts, resolver) match {
- case Some(outerAttr) => OuterReference(outerAttr)
- case None => u
- }
- } catch {
- case _: AnalysisException => u
+ val outer = outers.iterator
+ var expr = Option.empty[NamedExpression]
+ while (expr.isEmpty && outer.hasNext) {
+ expr = outer.next().resolve(nameParts, resolver)
+ }
+ expr match {
+ case Some(outerAttr) => OuterReference(outerAttr)
+ case None =>
--- End diff --
This seems to undo the resolution loop in `resolveSubQuery`. This does the
following: Resolve using the query -> resolve using the outer queries -> Fail
analysis. Are you sure this is sufficient to capture all resolution cases? I
cannot come up with a counter example (but give me time).
If we need the loop, then we need to modify this. If we don't then we
should update the `resolveSubQuery` function as well.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
