[
https://issues.apache.org/jira/browse/SPARK-60001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-60001:
-----------------------------------
Labels: pull-request-available (was: )
> ASOF JOIN fails inside a correlated subquery when its input reads the outer
> query. A regular JOIN works.
> --------------------------------------------------------------------------------------------------------
>
> Key: SPARK-60001
> URL: https://issues.apache.org/jira/browse/SPARK-60001
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.4.0
> Reporter: Luka Zdravic
> Priority: Minor
> Labels: pull-request-available
>
> {code:sql}
> SET spark.sql.join.asofJoin.enabled = true;
> CREATE OR REPLACE TEMPORARY VIEW events AS
> SELECT * FROM VALUES (1, 0), (3, 1), (6, 2), (8, 3) AS events(begin, value);
>
> -- fails:
> [UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.ACCESSING_OUTER_QUERY_COLUMN_IS_NOT_ALLOWED]
> SELECT begin, value IN (
> SELECT e1.value
> FROM (SELECT * FROM events e1 WHERE e1.value = events.value) e1
> ASOF JOIN range(1, 10) tbl(begin) MATCH_CONDITION (e1.begin >= tbl.begin)
> ) FROM events;
>
> -- works:
> SELECT begin, value IN (
> SELECT e1.value
> FROM (SELECT * FROM events e1 WHERE e1.value = events.value) e1
> JOIN range(1, 10) tbl(begin) ON e1.begin >= tbl.begin
> ) FROM events;
> {code}
> Cause: ValidateSubqueryExpression.checkPlan handles Join but not AsOfJoin.
> Expected: the ASOF query returns true for all 4 rows.
> More cases that fail the same way, while the same query with a regular JOIN
> works:
> - EXISTS and scalar subqueries
> - the outer reference in the right input of the ASOF JOIN
> - LEFT ASOF JOIN with the outer reference in the left input
> An outer reference in the ASOF JOIN ON condition fails later, in the
> optimizer: "Decorrelate inner query through AsOfJoin is not supported."
> {code:sql}
> SELECT begin, value IN (
> SELECT e1.value FROM events e1
> ASOF JOIN range(1, 10) tbl(begin) MATCH_CONDITION (e1.begin >= tbl.begin)
> ON e1.value = events.value
> ) FROM events;
> {code}
> Assigned to : [~lukazdravic]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
