Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/20402#discussion_r164030038
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -192,7 +192,7 @@ class Dataset[T] private[sql](
}
// Wraps analyzed logical plans with an analysis barrier so we won't
traverse/resolve it again.
- @transient private val planWithBarrier = AnalysisBarrier(logicalPlan)
+ @transient private val planWithBarrier =
AnalysisBarrier(EliminateBarriers(logicalPlan))
--- End diff --
I added eliminate barriers here to avoid excessive stacking of barriers.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
