allisonwang-db commented on code in PR #48820:
URL: https://github.com/apache/spark/pull/48820#discussion_r1858198370
##########
python/pyspark/sql/column.py:
##########
@@ -1524,7 +1524,11 @@ def over(self, window: "WindowSpec") -> "Column":
@dispatch_col_method
def outer(self) -> "Column":
"""
- Mark this column reference as an outer reference for subqueries.
+ Mark this as an outer Column if its expression may reference columns
in the outer plan.
+
+ This is used to trigger lazy analysis of Spark Classic DataFrame, so
that we can use it
+ to build subquery expressions. Spark Connect DataFrame is always
lazily analyzed and
+ does not need to use this function.
Review Comment:
I think we need more examples here (but can added in later PRs)
##########
python/pyspark/sql/column.py:
##########
@@ -1524,7 +1524,11 @@ def over(self, window: "WindowSpec") -> "Column":
@dispatch_col_method
def outer(self) -> "Column":
"""
- Mark this column reference as an outer reference for subqueries.
+ Mark this as an outer Column if its expression may reference columns
in the outer plan.
Review Comment:
```suggestion
Mark this column as an outer column if its expression refers to
columns from an outer query or plan
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
