Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17138#discussion_r104885896
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -118,6 +118,8 @@ abstract class Optimizer(sessionCatalog:
SessionCatalog, conf: CatalystConf)
SimplifyCreateMapOps) ::
Batch("Check Cartesian Products", Once,
CheckCartesianProducts(conf)) ::
+ Batch("Join Reorder", Once,
+ CostBasedJoinReorder(conf)) ::
--- End diff --
CostBasedJoinReorder can create two consecutive projections. The top
projection will have some expressions, and the bottom projection will only have
attributes (it is created by the join reorder rule). We can get rid of these
stacked projections by calling `CollapseProject` rule after the reordering.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
