[
https://issues.apache.org/jira/browse/SPARK-59529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luka Zdravic updated SPARK-59529:
---------------------------------
Description:
MATCH_CONDITION should require each operand to reference exactly one join input
(one side left, the other right). Spark enforces this for a wrong-side
reference but not for a no-reference operand: a literal, or a call like
current_date(), is accepted and pinned to its syntactic side, so the query
builds a real ASOF join with a constant operand.
– accepted, builds AsOfJoin (a >= 5), Inner:
SELECT * FROM (VALUES (1),(6)) l(a) ASOF JOIN (VALUES (10)) r(b)
MATCH_CONDITION (l.a >= 5);
– accepted:
SELECT * FROM (VALUES (DATE'2024-01-01')) l(a) ASOF JOIN (VALUES
(DATE'2024-01-02')) r(b)
MATCH_CONDITION (l.a >= current_date());
– already rejected today with ASOF_JOIN_MATCH_CONDITION_TABLE_REFERENCE (both
operands on one side):
SELECT * FROM (VALUES (1,2)) l(a,c) ASOF JOIN (VALUES (10)) r(b)
MATCH_CONDITION (l.a >= l.c);
Expected: reject a MATCH_CONDITION operand that references no join input,
reusing the existing ASOF_JOIN_MATCH_CONDITION_TABLE_REFERENCE error. The
check already exists for wrong-side references; extend it to the no-reference
case.
Impact: a query that reads like an as-of join builds with a constant operand
and no defined nearest-match.
Umbrella: SPARK-59526
was:
MATCH_CONDITION should require each operand to reference exactly one join input
(one side left, the other right). Spark enforces this for a wrong-side
reference but not for a no-reference operand: a literal, or a call like
current_date(), is accepted and pinned to its syntactic side, so the query
builds a real ASOF join with a constant operand.
Confirmed on commit 083d824, ANSI off and on:
-- accepted, builds AsOfJoin (a >= 5), Inner:
SELECT * FROM (VALUES (1),(6)) l(a) ASOF JOIN (VALUES (10)) r(b)
MATCH_CONDITION (l.a >= 5);
-- accepted:
SELECT * FROM (VALUES (DATE'2024-01-01')) l(a) ASOF JOIN (VALUES
(DATE'2024-01-02')) r(b)
MATCH_CONDITION (l.a >= current_date());
-- already rejected today with ASOF_JOIN_MATCH_CONDITION_TABLE_REFERENCE (both
operands on one side):
SELECT * FROM (VALUES (1,2)) l(a,c) ASOF JOIN (VALUES (10)) r(b)
MATCH_CONDITION (l.a >= l.c);
Expected: reject a MATCH_CONDITION operand that references no join input,
reusing the existing ASOF_JOIN_MATCH_CONDITION_TABLE_REFERENCE error. The
check already exists for wrong-side references; extend it to the no-reference
case.
Impact: a query that reads like an as-of join builds with a constant operand
and no defined nearest-match. Low priority.
Umbrella: SPARK-59526
> ASOF JOIN MATCH_CONDITION accepts a constant operand referencing no join input
> ------------------------------------------------------------------------------
>
> Key: SPARK-59529
> URL: https://issues.apache.org/jira/browse/SPARK-59529
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Luka Zdravic
> Priority: Minor
>
> MATCH_CONDITION should require each operand to reference exactly one join
> input (one side left, the other right). Spark enforces this for a wrong-side
> reference but not for a no-reference operand: a literal, or a call like
> current_date(), is accepted and pinned to its syntactic side, so the query
> builds a real ASOF join with a constant operand.
> – accepted, builds AsOfJoin (a >= 5), Inner:
> SELECT * FROM (VALUES (1),(6)) l(a) ASOF JOIN (VALUES (10)) r(b)
> MATCH_CONDITION (l.a >= 5);
> – accepted:
> SELECT * FROM (VALUES (DATE'2024-01-01')) l(a) ASOF JOIN (VALUES
> (DATE'2024-01-02')) r(b)
> MATCH_CONDITION (l.a >= current_date());
> – already rejected today with ASOF_JOIN_MATCH_CONDITION_TABLE_REFERENCE (both
> operands on one side):
> SELECT * FROM (VALUES (1,2)) l(a,c) ASOF JOIN (VALUES (10)) r(b)
> MATCH_CONDITION (l.a >= l.c);
> Expected: reject a MATCH_CONDITION operand that references no join input,
> reusing the existing ASOF_JOIN_MATCH_CONDITION_TABLE_REFERENCE error. The
> check already exists for wrong-side references; extend it to the no-reference
> case.
> Impact: a query that reads like an as-of join builds with a constant operand
> and no defined nearest-match.
> Umbrella: SPARK-59526
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
