beliefer commented on code in PR #43449:
URL: https://github.com/apache/spark/pull/43449#discussion_r1366682188
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/InjectRuntimeFilter.scala:
##########
@@ -305,7 +305,8 @@ object InjectRuntimeFilter extends Rule[LogicalPlan] with
PredicateHelper with J
}
private def findBloomFilterWithKey(plan: LogicalPlan, key: Expression):
Boolean = {
- plan.exists {
+ // Ensure that sibling nodes under the same Join do not have runtime filter
+ !plan.exists(_.isInstanceOf[Join]) && plan.exists {
Review Comment:
Now, the support are multiple levels. If there are two nested joins that
require shuffling, when transforming up, runtime filter will first be inserted
at one end of the lower level join. When scrolling up to the upper join, the
lower runtime filter can often be inserted into one end of the upper join.
For example, `bigA join B join BigC where a=b and b=c` cannot ignore the
opportunity to insert runtime filter into BigC just because B has inserted a
runtime filter into bigA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
