Luka Zdravic created SPARK-59438:
------------------------------------
Summary: Add planner-stage test coverage for ASOF JOIN sort-merge
operator
Key: SPARK-59438
URL: https://issues.apache.org/jira/browse/SPARK-59438
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 5.0.0
Reporter: Luka Zdravic
The planner-stage path for ASOF JOIN — AsOfJoinSelection
(SparkStrategies.scala) selecting SortMergeAsOfJoinExec — is exercised
end-to-end by the SQL golden suite (join-asof-*.sql) and by
SortMergeAsOfJoinSuite / AsOfJoinSortMergeSQLSuite.
Two planner-stage behaviors, however, have no test:
# Physical plan shape (required distribution/ordering). SortMergeAsOfJoinExec
requires AllTuples (a single partition) when there are no equi-keys, and
clustered+sorted children otherwise. No test asserts the operator's physical
requirements: that a no-equi-key ASOF forces a single-partition exchange, and
that no redundant shuffle/sort is added when the child already satisfies them.
Sibling joins cover this in PlannerSuite; ASOF has nothing there.
# EqualNullSafe (<=>) in ON. The strategy deliberately excludes <=> from the
equi-keys and routes it to the residual condition. The resulting behavior —
NULL keys do match (unlike =), and with no equi-key both sides collapse to one
partition — is untested. (The = counterpart, "NULL never matches", is covered.)
This ticket adds the missing planner-stage tests. No production behavior
changes.
UMBRELLA: SPARK-59298
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
