zhengruifeng commented on PR #55947:
URL: https://github.com/apache/spark/pull/55947#issuecomment-4518796291
@cloud-fan this PR is not just for parity (thought some tests behave the
same on both connect and classic).
E.g.
```
def test_resolve_after_agg_alias_shadow(self):
# An aggregate output named `c` via alias() collides by name with
# the source `c`. The tagged cdf["c"] still references the source
# attribute that has been aggregated away.
# Classic: fails.
sdf = self.spark.sql("SELECT 1 AS x")
with self.assertRaises(AnalysisException):
sdf.groupBy().agg(SF.sum("x").alias("c")).select(sdf["c"]).collect()
# Connect strict: fails.
with
self.connect_conf({"spark.sql.analyzer.strictDataFrameColumnResolution": True}):
cdf = self.connect.sql("SELECT 1 AS x")
with self.assertRaisesRegex(AnalysisException,
"CANNOT_RESOLVE_DATAFRAME_COLUMN"):
cdf.groupBy().agg(CF.sum("x").alias("c")).select(cdf["c"]).collect()
# Connect lenient: succeeds via name-based fallback.
with
self.connect_conf({"spark.sql.analyzer.strictDataFrameColumnResolution":
False}):
cdf = self.connect.sql("SELECT 1 AS x")
cdf.groupBy().agg(CF.sum("x").alias("c")).select(cdf["c"]).collect()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
