agrawaldevesh commented on a change in pull request #29304:
URL: https://github.com/apache/spark/pull/29304#discussion_r464101354
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala
##########
@@ -245,7 +244,7 @@ case class BroadcastHashJoinExec(
|boolean $found = false;
|// generate join key for stream side
|${keyEv.code}
- |if ($anyNull) {
+ |if (${if (isLongHashedRelation) s"$anyNull" else
s"${keyEv.value}.allNull()"}) {
Review comment:
Thanks for digging this, I think multi-key NAAJ would preclude the
packing of multiple keys into a single long key unless of all of those sub-keys
are actually non nullable ?
I think it would actually be incorrect to NOT use the key explosion of
HashedRelation when there are multiple logical keys that can be independently
nullable. Is there a way we can 'undo' the packing into the LongHashedRelation
and switch back to a regular HashedRelation ?
Is there a test case for this scenario of multiple nullable integral keys ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
