ulysses-you commented on code in PR #40578:
URL: https://github.com/apache/spark/pull/40578#discussion_r1151406262
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala:
##########
@@ -132,23 +132,19 @@ case class BroadcastHashJoinExec(
} else if (hashed == HashedRelationWithAllNullKeys) {
Iterator.empty
} else {
- val keyGenerator = UnsafeProjection.create(
- BindReferences.bindReferences[Expression](
- leftKeys,
- AttributeSeq(left.output))
- )
- streamedIter.filter(row => {
+ val keyGenerator = streamSideKeyGenerator()
Review Comment:
+1, it acutally prevents the potential issue that all keys in shuffle join
should rewrite as LongType if possible. Previous, the hash relation has
rewritten but the probe key did not. It's not a bug as NAAJ now only supports
one column.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
