cloud-fan commented on code in PR #42317:
URL: https://github.com/apache/spark/pull/42317#discussion_r1360437686
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/InjectRuntimeFilter.scala:
##########
@@ -118,17 +122,34 @@ object InjectRuntimeFilter extends Rule[LogicalPlan] with
PredicateHelper with J
* filter should be selective and the filter condition (including
expressions in the child
* plan referenced by the filter condition) should be a simple expression,
so that we do
* not add a subquery that might have an expensive computation.
+ *
+ * Note: The parameter `filterCreationSideExp` represents the creation side
expression initially
+ * confirmed outside the extract method. The target creation side expression
may change during
+ * the join relationship pass process. So the extract method returns a
optional tuple,
+ * which contains the target expression and plan of creation side.
*/
private def extractSelectiveFilterOverScan(
plan: LogicalPlan,
- filterCreationSideExp: Expression): Option[LogicalPlan] = {
- @tailrec
+ filterCreationSideExp: Expression): Option[(Expression, LogicalPlan)] = {
+
+ /**
+ * Extracts a sub-plan which is a simple filter over scan from the input
plan iteratively
+ * by the way show below:
+ * - Extracts a sub-plan from a plan without join nodes.
+ * - Extracts a sub-plan from left or right child of join nodes.
+ *
+ * Note: There are two situations if extracts a sub-plan from any child of
join nodes.
+ * If we can extract a sub-plan from one child of join node, using it
directly.
+ * Otherwise, we extract a sub-plan from another child of join node if
+ * we can find out the passed creation side expression by the join
relationship pass process.
+ */
def extract(
p: LogicalPlan,
predicateReference: AttributeSet,
hasHitFilter: Boolean,
hasHitSelectiveFilter: Boolean,
- currentPlan: LogicalPlan): Option[LogicalPlan] = p match {
+ targetPlan: LogicalPlan,
+ targetCreationSideExpr: Expression): Option[(Expression, LogicalPlan)]
= p match {
Review Comment:
let's name it `targetKey`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
