Cheng Su created SPARK-32420:
--------------------------------
Summary: Add handling for unique key in non-codegen hash join
Key: SPARK-32420
URL: https://issues.apache.org/jira/browse/SPARK-32420
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.1.0
Reporter: Cheng Su
`HashRelation` has two separate code paths for unique key look up and
non-unique key look up E.g. in its subclass
`UnsafeHashedRelation`([https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala#L144-L177]),
unique key look up is more efficient as it does not have extra
`Iterator[UnsafeRow].hasNext()/next()` overhead per row.
`BroadcastHashJoinExec` has handled unique key vs non-unique key separately in
code-gen path
([https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala#L289-L321]).
But the non-codegen path for broadcast hash join and shuffled hash join do not
separate it yet, so adding the support here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
