LukaZdravic opened a new pull request, #58853:
URL: https://github.com/apache/spark/pull/58853
### What changes were proposed in this pull request?
Make SQL `ASOF JOIN ... MATCH_CONDITION (a op b)` reject an operand that
references no join input.
`MATCH_CONDITION` requires each operand to reference exactly one join input:
one operand from the left side and the other from the right.
`AsOfJoin.operandJoinSide` decides an operand's side from its attribute
references:
- references only the left input -> left
- references only the right input -> right
- references the wrong side or both sides -> rejected with
`ASOF_JOIN_MATCH_CONDITION_TABLE_REFERENCE`
- references no input (a literal, or a call like `current_date()`) ->
**previously pinned to its syntactic side and accepted** (the bug)
This PR changes `operandJoinSide` to return `None` for a no-reference
operand. A `None` side falls through to the existing
`ASOF_JOIN_MATCH_CONDITION_TABLE_REFERENCE` error in
`normalizeMatchOperands`, the same error already raised for wrong-side
references. The fixed-point analyzer (`ResolveAsOfJoin`) and the single-pass
analyzer (`AsOfJoinResolver`) share `normalizeMatchOperands`, so both behave
identically.
### Why are the changes needed?
A query that reads like an as-of join could build a real ASOF join with a
constant operand and no defined nearest-match. Both of these were accepted
before this PR and build an `AsOfJoin`:
```sql
SELECT * FROM (VALUES (1),(6)) l(a) ASOF JOIN (VALUES (10)) r(b)
MATCH_CONDITION (l.a >= 5);
SELECT * FROM (VALUES (DATE'2024-01-01')) l(a) ASOF JOIN (VALUES
(DATE'2024-01-02')) r(b)
MATCH_CONDITION (l.a >= current_date());
```
The equivalent wrong-side case was already rejected:
```sql
SELECT * FROM (VALUES (1,2)) l(a,c) ASOF JOIN (VALUES (10)) r(b)
MATCH_CONDITION (l.a >= l.c); -- ASOF_JOIN_MATCH_CONDITION_TABLE_REFERENCE
```
This PR makes the no-reference case consistent with the wrong-side case.
### Does this PR introduce _any_ user-facing change?
Yes, but only within the unreleased line. SQL `ASOF JOIN` is a new,
unreleased feature (`spark.sql.join.asofJoin.enabled`, added in 4.3.0, off by
default). A `MATCH_CONDITION` operand that references no join input now fails
analysis with `ASOF_JOIN_MATCH_CONDITION_TABLE_REFERENCE` (SQLSTATE 42K0E)
instead of being accepted. No released version is affected.
### How was this patch tested?
- `AsOfJoinSQLSuite`: the two tests that asserted constant operands are
accepted now assert the rejection error via `checkError` (a literal operand,
and the query-foldable `current_timestamp()`).
- Golden files: added FVT-ASOF-3-024/025/026 in `join-asof-errors.sql`
(literal right, literal left, `current_timestamp()`); removed the old accept
cases FVT-ASOF-8-008* from `join-asof-expressions.sql` and left a pointer
comment; regenerated the golden `.sql.out` files.
- Ran `AsOfJoinSQLSuite` and `AsOfJoinSortMergeSQLSuite`: 47 tests, all pass.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
