dongjoon-hyun commented on a change in pull request #32179:
URL: https://github.com/apache/spark/pull/32179#discussion_r615441050
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
##########
@@ -949,22 +1007,17 @@ trait CheckAnalysis extends PredicateHelper with
LookupCatalog {
// The other operator is Join. Filter can be anywhere in a correlated
subquery.
case f: Filter =>
val (correlated, _) =
splitConjunctivePredicates(f.condition).partition(containsOuter)
-
- // Find any non-equality correlated predicates
- foundNonEqualCorrelatedPred = foundNonEqualCorrelatedPred ||
correlated.exists {
- case _: EqualTo | _: EqualNullSafe => false
- case _ => true
- }
+ unsupportedPredicates ++= correlated.filter(isUnsupportedPredicate)
failOnInvalidOuterReference(f)
// Aggregate cannot host any correlated expressions
// It can be on a correlation path if the correlation contains
- // only equality correlated predicates.
+ // only supported correlated equality predicates.
// It cannot be on a correlation path if the correlation has
// non-equality correlated predicates.
case a: Aggregate =>
failOnInvalidOuterReference(a)
- failOnNonEqualCorrelatedPredicate(foundNonEqualCorrelatedPred, a)
+ failOnUnsupportedCorrelatedPredicate(unsupportedPredicates, a)
Review comment:
This seems to cause a compilation error with Scala 2.13. Could you
re-check with Scala 2.13, @allisonwang-db ?
```
[error]
/home/runner/work/spark/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala:1020:46:
type mismatch;
[error] found :
scala.collection.mutable.ArrayBuffer[org.apache.spark.sql.catalyst.expressions.Expression]
[error] required: Seq[org.apache.spark.sql.catalyst.expressions.Expression]
[error] failOnUnsupportedCorrelatedPredicate(unsupportedPredicates,
a)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
