Allison Wang created SPARK-38155:
------------------------------------
Summary: Disallow distinct aggregate in lateral subqueries with
unsupported correlated predicates
Key: SPARK-38155
URL: https://issues.apache.org/jira/browse/SPARK-38155
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 3.3.0
Reporter: Allison Wang
Block lateral subqueries in CheckAnalysis that contains DISTINCT aggregate and
correlated non-equality predicates. This can lead to incorrect results as
DISTINCT will be rewritten as Aggregate during the optimization phase.
For example
CREATE VIEW t1(c1, c2) AS VALUES (0, 1)
CREATE VIEW t2(c1, c2) AS VALUES (1, 2), (2, 2)
SELECT * FROM t1 JOIN LATERAL (SELECT DISTINCT c2 FROM t2 WHERE c1 > t1.c1)
The correct results should be (0, 1, 2) but currently, it gives [(0, 1, 2),
(0, 1, 2)]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
