Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21529
I was surprised that this bug is present even if the table is not bucketed,
and then I found out another problem in the code: `reorderJoinPredicates`
transform the plan again. `reorderJoinPredicates` is called during plan
transformation and we should not double transform the plan.
This makes the bug has a much larger impact: when we do a shuffle join, we
add shuffle exchange to both of the join sides, which then triggers
`reorderJoinPredicates` because of the double transformation problem.
@mgaido91 let's also remove the `transformUp` in `reorderJoinPredicates`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
