[
https://issues.apache.org/jira/browse/SPARK-60001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luka Zdravic updated SPARK-60001:
---------------------------------
Description:
{code:sql}
SET spark.sql.join.asofJoin.enabled = true;
CREATE OR REPLACE TEMPORARY VIEW events AS
SELECT * FROM VALUES (1, 0), (3, 1), (6, 2), (8, 3) AS events(begin, value);
-- fails:
[UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.ACCESSING_OUTER_QUERY_COLUMN_IS_NOT_ALLOWED]
SELECT begin, value IN (
SELECT e1.value
FROM (SELECT * FROM events e1 WHERE e1.value = events.value) e1
ASOF JOIN range(1, 10) tbl(begin) MATCH_CONDITION (e1.begin >= tbl.begin)
) FROM events;
-- works:
SELECT begin, value IN (
SELECT e1.value
FROM (SELECT * FROM events e1 WHERE e1.value = events.value) e1
JOIN range(1, 10) tbl(begin) ON e1.begin >= tbl.begin
) FROM events;
{code}
Cause: ValidateSubqueryExpression.checkPlan handles Join but not AsOfJoin.
Expected: the ASOF query returns true for all 4 rows.
Assigned to : [~lukazdravic]
was:
{code:sql}
SET spark.sql.join.asofJoin.enabled = true;
CREATE OR REPLACE TEMPORARY VIEW events AS
SELECT * FROM VALUES (1, 0), (3, 1), (6, 2), (8, 3) AS events(begin, value);
-- fails:
[UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.ACCESSING_OUTER_QUERY_COLUMN_IS_NOT_ALLOWED]
SELECT begin, value IN (
SELECT e1.value
FROM (SELECT * FROM events e1 WHERE e1.value = events.value) e1
ASOF JOIN range(1, 10) tbl(begin) MATCH_CONDITION (e1.begin >= tbl.begin)
) FROM events;
-- works:
SELECT begin, value IN (
SELECT e1.value
FROM (SELECT * FROM events e1 WHERE e1.value = events.value) e1
JOIN range(1, 10) tbl(begin) ON e1.begin >= tbl.begin
) FROM events;
{code}
Cause: ValidateSubqueryExpression.checkPlan handles Join but not AsOfJoin.
Expected: the ASOF query returns true for all 4 rows.
> ASOF JOIN fails inside a correlated subquery when its input reads the outer
> query. A regular JOIN works.
> --------------------------------------------------------------------------------------------------------
>
> Key: SPARK-60001
> URL: https://issues.apache.org/jira/browse/SPARK-60001
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.4.0
> Reporter: Luka Zdravic
> Priority: Minor
>
> {code:sql}
> SET spark.sql.join.asofJoin.enabled = true;
> CREATE OR REPLACE TEMPORARY VIEW events AS
> SELECT * FROM VALUES (1, 0), (3, 1), (6, 2), (8, 3) AS events(begin, value);
>
> -- fails:
> [UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.ACCESSING_OUTER_QUERY_COLUMN_IS_NOT_ALLOWED]
> SELECT begin, value IN (
> SELECT e1.value
> FROM (SELECT * FROM events e1 WHERE e1.value = events.value) e1
> ASOF JOIN range(1, 10) tbl(begin) MATCH_CONDITION (e1.begin >= tbl.begin)
> ) FROM events;
>
> -- works:
> SELECT begin, value IN (
> SELECT e1.value
> FROM (SELECT * FROM events e1 WHERE e1.value = events.value) e1
> JOIN range(1, 10) tbl(begin) ON e1.begin >= tbl.begin
> ) FROM events;
> {code}
> Cause: ValidateSubqueryExpression.checkPlan handles Join but not AsOfJoin.
> Expected: the ASOF query returns true for all 4 rows.
> Assigned to : [~lukazdravic]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
