Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/21822#discussion_r204166360
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
---
@@ -533,7 +537,8 @@ trait CheckAnalysis extends PredicateHelper {
// Simplify the predicates before validating any unsupported
correlation patterns
// in the plan.
- BooleanSimplification(sub).foreachUp {
+ // TODO(rxin): Why did this need to call BooleanSimplification???
--- End diff --
@hvanhovell Hi Herman, as you said, we do the actual pulling up of the
predicates in the optimizer in PullupCorrelatedPredicates in subquery.scala. We
are also doing a BooleanSimplication first before traversing the plan there. In
here, we are doing the error reporting and i thought it would be better to keep
the traversal the same way. Basically previously we did the error reporting and
rewriting in Analyzer and now, we do the error reporting in checkAnalysis and
rewriting in Optimizer. Just to refresh your memory so you can help to take the
right call here :-)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
