Github user srinathshankar commented on a diff in the pull request:
https://github.com/apache/spark/pull/15238#discussion_r80593053
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -579,8 +579,13 @@ object InferFiltersFromConstraints extends
Rule[LogicalPlan] with PredicateHelpe
* Combines all adjacent [[Union]] operators into a single [[Union]].
*/
object CombineUnions extends Rule[LogicalPlan] {
- def apply(plan: LogicalPlan): LogicalPlan = plan transform {
- case Unions(children) => Union(children)
+ def apply(plan: LogicalPlan): LogicalPlan = plan transformDown {
+ case Unions(children, isDistinct) =>
--- End diff --
Could we do Unions(children, true) and Unions(children, false) as separate
case statements, that is separate pattern matching ? That would make it more
clear.
Like a UnionAll and a UnionDistinct ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
