maryannxue commented on a change in pull request #24692: [SPARK-27824][SQL]
Make rule EliminateResolvedHint idempotent
URL: https://github.com/apache/spark/pull/24692#discussion_r287468312
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/EliminateResolvedHint.scala
##########
@@ -29,7 +29,7 @@ object EliminateResolvedHint extends Rule[LogicalPlan] {
// is using transformUp rather than resolveOperators.
def apply(plan: LogicalPlan): LogicalPlan = {
val pulledUp = plan transformUp {
- case j: Join =>
+ case j: Join if j.hint == JoinHint.NONE =>
Review comment:
I think the "best" way to fix it is to merge the new hints into the existing
`JoinHint`, but the question is which one should take precedence, the new one
or the existing one?
So far, if things all work right, we should never come to a point where
there is a non-empty new hint and a non-empty existing hint, that is, the hint
has either been pulled into the join node or it hasn't been processed it. In
either case, this fix is sufficient.
The removal of the `ResolvedHint` is taken care of by the following lines
and should not happen at all if the rule has been applied once.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
