Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/19452#discussion_r144437093
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingSymmetricHashJoinExec.scala
---
@@ -161,6 +164,10 @@ case class StreamingSymmetricHashJoinExec(
new SerializableConfiguration(SessionState.newHadoopConf(
sparkContext.hadoopConfiguration, sqlContext.conf)))
+
+ val nullLeft = new
GenericInternalRow(left.output.map(_.withNullability(true)).length)
+ val nullRight = new
GenericInternalRow(right.output.map(_.withNullability(true)).length)
--- End diff --
This can also be moved into the Joiner. Use `Joiner.inputAttributes`,
instead of `left.output`/`right.output`
Basically if you have to define variables outside the Joine with the name
"left", then it highly likely that you can move into the Joiner, as it is meant
capture all the context needed to process one side.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
