Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/20094#discussion_r158879283
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -665,14 +664,18 @@ class Analyzer(
* Generate a new logical plan for the right child with different
expression IDs
* for all conflicting attributes.
*/
- private def dedupRight (left: LogicalPlan, originalRight:
LogicalPlan): LogicalPlan = {
- // Remove analysis barrier if any.
- val right = EliminateBarriers(originalRight)
--- End diff --
If right plan is wrapped (e.g., we join two datasets) in an analysis
barrier, the later `right.collect` doesn't work.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
