Luka Zdravic created SPARK-59409:
------------------------------------
Summary: Expand RewriteAsOfJoin optimizer rule test coverage
Key: SPARK-59409
URL: https://issues.apache.org/jira/browse/SPARK-59409
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 5.0.0
Reporter: Luka Zdravic
RewriteAsOfJoinSuite (the unit test for the RewriteAsOfJoin optimizer rule)
previously contained only positive rewrite tests and left several parts of the
rule uncovered. This is a test-only change that closes those gaps and reduces
duplication.
Gaps closed:
* One (direction, tolerance, allowExactMatches) combination was missing:
nearest + allowExactMatches = false + no tolerance, whose as-of condition is
NOT(left = right).
* The "left outer" test actually built an Inner join (a duplicate of
"simple") and never tested LeftOuter; it now builds a real LeftOuter join,
isolating the one thing LeftOuter changes (omitting the IS NOT NULL filter).
* The rule's no-op guard
!conf.useSortMergeAsOfJoinOperator(requiresSortMergeAsOfJoin) had no coverage.
Added negative tests for both triggers: the requiresSortMergeAsOfJoin node
flag, and the sort-merge as-of config.
* The attribute remapping done by transformUpWithNewOutput was never
exercised (all tests put AsOfJoin at the plan root). Added a test with a
Project above the join.
Refactor:
* Introduced a shared expectedRewrite(filter, orderExpression, joinType)
helper so each test spells out only the parts that vary, removing ~15 lines of
duplicated plan-building per test. Mirrors the sibling
RewriteNearestByJoinSuite.
No production code changes; no user-facing change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
