Luka Zdravic created SPARK-59528:
------------------------------------
Summary: ASOF JOIN MATCH_CONDITION rejects arrays with coercible
element types
Key: SPARK-59528
URL: https://issues.apache.org/jira/browse/SPARK-59528
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 5.0.0
Reporter: Luka Zdravic
Inside a MATCH_CONDITION, comparing array<int> to array<long> fails with
ASOF_JOIN_MATCH_CONDITION_INVALID_TYPE, because ASOF requires identical array
element types. The same comparison outside ASOF works: Spark coerces the
element types. ASOF should apply the same coercion.
-- fails:
SELECT * FROM (VALUES (array(1))) l(a)
ASOF JOIN (VALUES (array(CAST(1 AS BIGINT)))) r(b)
MATCH_CONDITION (l.a >= r.b); -- ASOF_JOIN_MATCH_CONDITION_INVALID_TYPE
-- works:
SELECT array(1) >= array(CAST(1 AS BIGINT)); -- true
Expected: MATCH_CONDITION accepts arrays whose element types differ but are
coercible, same as the comparison operator.
Reproduces under ANSI on and off.
Impact: a valid array as-of condition that is a legal comparison cannot be
written.
Umbrella: SPARK-59526
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
