Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22205#discussion_r212894435
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -130,6 +130,10 @@ abstract class Optimizer(sessionCatalog:
SessionCatalog)
// since the other rules might make two separate Unions operators
adjacent.
Batch("Union", Once,
CombineUnions) ::
+ // run this once earlier. this might simplify the plan and reduce cost
of optimizer
--- End diff --
Hi @bogdanrdc , could explain more about this? Why is it necessary?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
