bmarcott commented on issue #27096: [SPARK-28148][SQL] Repartition after join is not optimized away URL: https://github.com/apache/spark/pull/27096#issuecomment-574005977 i'll update soon with the new rule `PruneShuffleAndSort` I'm iffy on including [PR-26946](https://github.com/apache/spark/pull/26946). `EnsureRequirements` should probably only introduce shuffles/ordering if it needs to, rather than adding blindly and relying on later rule to prune. Or maybe it is better to include so it'll apply to any case (haven't thought deeply to the cases here)?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
