RussellSpitzer commented on issue #2533:
URL: https://github.com/apache/iceberg/issues/2533#issuecomment-834940896
Spark normally just won't plan a join when one side is empty, quick
demonstration
```
scala> val left = spark.createDataset(Seq.empty[Row])
left: org.apache.spark.sql.Dataset[Row] = [i: int]
scala> val right = spark.createDataset(Seq(Row(1)))
right: org.apache.spark.sql.Dataset[Row] = [i: int]
scala> left.join(right, left("i") === right("i"), "left_outer").show
+---+---+
| i| i|
+---+---+
+---+---+
scala> left.join(right, left("i") === right("i"),
"left_outer").queryExecution.logical
res14: org.apache.spark.sql.catalyst.plans.logical.LogicalPlan =
Join LeftOuter, (i#133 = i#137)
:- LocalRelation <empty>, [i#133]
+- LocalRelation [i#137]
scala> left.join(right, left("i") === right("i"),
"left_outer").queryExecution.spark
sparkPlan sparkSession
scala> left.join(right, left("i") === right("i"),
"left_outer").queryExecution.sparkPlan
res15: org.apache.spark.sql.execution.SparkPlan =
LocalTableScan <empty>, [i#133, i#137]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
