Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/20345#discussion_r175696187
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala
---
@@ -172,17 +174,23 @@ object ExtractFiltersAndInnerJoins extends
PredicateHelper {
case Filter(filterCondition, j @ Join(left, right, _: InnerLike,
joinCondition)) =>
val (plans, conditions) = flattenJoin(j)
(plans, conditions ++ splitConjunctivePredicates(filterCondition))
-
+ case p @ Project(_, j @ Join(left, right, _: InnerLike,
joinCondition)) =>
+ // Keep flattening joins when projects having attributes only
+ if (p.outputSet.subsetOf(j.outputSet)) {
--- End diff --
If we want to make sure the project has attributes only, should it be
`p.projectList.forall(_.isInstanceOf[Attribute])`?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
