allisonwang-db commented on PR #48820:
URL: https://github.com/apache/spark/pull/48820#issuecomment-2482626978
I think there is another issue with the `outer` API. The `where` or
`filter` API can take a SQL expression, but we don't have a SQL API for outer
reference. So users can't really use SQL expressions together with the subquery
API:
```python
# SELECT a, (SELECT sum(d) FROM r WHERE a = c) FROM l
l.select(
"a",
(
r
.where("a = c") <- SQL expression won't work and no alternative to
make this analysis lazy
.select(sf.sum("d"))
.scalar()
),
).show()
```
And this is giving
```
AnalysisException: [UNRESOLVED_COLUMN.WITH_SUGGESTION] A column, variable,
or function parameter with name `a` cannot be resolved. Did you mean one of the
following? [`c`, `d`]. SQLSTATE: 42703; line 1 pos 0;
'Filter ('a = c#2L)
+- LogicalRDD [c#2L, d#3], false
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
