navinvishy commented on a change in pull request #29170:
URL: https://github.com/apache/spark/pull/29170#discussion_r519534928
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##########
@@ -116,7 +116,8 @@ abstract class Optimizer(catalogManager: CatalogManager)
operatorOptimizationRuleSet.filterNot(_ == InferFiltersFromConstraints)
Batch("Operator Optimization before Inferring Filters", fixedPoint,
rulesWithoutInferFiltersFromConstraints: _*) ::
- Batch("Infer Filters", Once,
+ Batch("Infer Filters", fixedPoint,
+ PushDownPredicates,
Review comment:
I initially looked at trying to see if InferFiltersFromConstraints could
itself handle this, but I think I could not figure out a way to solve the case
in the unit test. I also don't think that this can cause an infinite loop, but
let me take another look at this to see if we can either be sure that it cannot
result in an infinite loop or if we can push this into
InferFiltersFromConstraints so that we don't need the fixedPoint.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
