maryannxue commented on a change in pull request #23036: [SPARK-26065][SQL]
Change query hint from a `LogicalPlan` to a field
URL: https://github.com/apache/spark/pull/23036#discussion_r245355183
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
##########
@@ -43,10 +43,15 @@ object ReorderJoin extends Rule[LogicalPlan] with
PredicateHelper {
*
* @param input a list of LogicalPlans to inner join and the type of inner
join.
* @param conditions a list of condition for join.
+ * @param leftPlans a list of LogicalPlans contained in the left join child.
+ * @param hintMap a map of relations to their corresponding hints.
*/
@tailrec
- final def createOrderedJoin(input: Seq[(LogicalPlan, InnerLike)],
conditions: Seq[Expression])
- : LogicalPlan = {
+ final def createOrderedJoin(
+ input: Seq[(LogicalPlan, InnerLike)],
+ conditions: Seq[Expression],
+ leftPlans: Seq[LogicalPlan],
+ hintMap: Map[Seq[LogicalPlan], HintInfo]): LogicalPlan = {
Review comment:
After ReorderJoin, new conditions might be pushed into a join relation. For
example, in
https://github.com/apache/spark/pull/23036/files#diff-fb10f33381c6d7cc8bfbde63d7f2c557R109,
the join order has remained the same, but the first join between "a" and "b"
now has a new condition, as "a.a1 = b.b1". I'd still wanna treat it as the same
join, thus retaining its hint. If we were to compare the join LogicalPlan, they
would not match. Since ReorderJoin is simply dealing with left-deep trees, as
long as seq of join child relations are fixed, the join order is fixed too. So
we can compare the seq of join child relations instead, in order to accommodate
this "new conditions pushed down" situation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
