[
https://issues.apache.org/jira/browse/SPARK-31377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Srinivas Rishindra Pothireddi updated SPARK-31377:
--------------------------------------------------
Description:
For some combinations of join algorithm and join types there are no unit tests
for the "number of output rows" metric.
A list of missing unit tests include the following.
* SortMergeJoin: ExistenceJoin
* ShuffledHashJoin: OuterJoin, ReftOuter, RightOuter, LeftAnti, LeftSemi,
ExistenseJoin
* BroadcastNestedLoopJoin: RightOuter, ExistenceJoin, InnerJoin
* BroadcastHashJoin: LeftAnti, ExistenceJoin
was:
For some combinations of join algorithm and join types there are no unit tests
for the "number of output rows" metric.
For Inner join in SortMergeJoin there is code in unit tests with code
generation enabled. There are no tests for the "number of output rows" metric
with code generation disabled.
A list of missing unit tests include the following.
* SortMergeJoin: ExistenceJoin
* ShuffledHashJoin: OuterJoin, ReftOuter, RightOuter, LeftAnti, LeftSemi,
ExistenseJoin
* BroadcastNestedLoopJoin: RightOuter, ExistenceJoin, InnerJoin
* BroadcastHashJoin: LeftAnti, ExistenceJoin
* For inner join in SortMergeJoin with code generation turned off.
> Add missing unit tests for "number of output rows" metric for joins in
> SQLMetricsSuite
> --------------------------------------------------------------------------------------
>
> Key: SPARK-31377
> URL: https://issues.apache.org/jira/browse/SPARK-31377
> Project: Spark
> Issue Type: Improvement
> Components: SQL, Tests
> Affects Versions: 3.0.0
> Reporter: Srinivas Rishindra Pothireddi
> Priority: Major
>
> For some combinations of join algorithm and join types there are no unit
> tests for the "number of output rows" metric.
> A list of missing unit tests include the following.
> * SortMergeJoin: ExistenceJoin
> * ShuffledHashJoin: OuterJoin, ReftOuter, RightOuter, LeftAnti, LeftSemi,
> ExistenseJoin
> * BroadcastNestedLoopJoin: RightOuter, ExistenceJoin, InnerJoin
> * BroadcastHashJoin: LeftAnti, ExistenceJoin
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
