Luka Zdravic created SPARK-60001:
------------------------------------
Summary: ASOF JOIN fails inside a correlated subquery when its
input reads the outer query. A regular JOIN works.
Key: SPARK-60001
URL: https://issues.apache.org/jira/browse/SPARK-60001
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.4.0
Reporter: Luka Zdravic
{code:sql}
SET spark.sql.join.asofJoin.enabled = true;
CREATE OR REPLACE TEMPORARY VIEW events AS
SELECT * FROM VALUES (1, 0), (3, 1), (6, 2), (8, 3) AS events(begin, value);
-- fails:
[UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.ACCESSING_OUTER_QUERY_COLUMN_IS_NOT_ALLOWED]
SELECT begin, value IN (
SELECT e1.value
FROM (SELECT * FROM events e1 WHERE e1.value = events.value) e1
ASOF JOIN range(1, 10) tbl(begin) MATCH_CONDITION (e1.begin >= tbl.begin)
) FROM events;
-- works:
SELECT begin, value IN (
SELECT e1.value
FROM (SELECT * FROM events e1 WHERE e1.value = events.value) e1
JOIN range(1, 10) tbl(begin) ON e1.begin >= tbl.begin
) FROM events;
{code}
Cause: ValidateSubqueryExpression.checkPlan handles Join but not AsOfJoin.
Expected: the ASOF query returns true for all 4 rows.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
