ekoifman commented on pull request #31653: URL: https://github.com/apache/spark/pull/31653#issuecomment-802510977
I wanted to have a call because there was disconnect about how the patch is structured. If you now agree that `OptimizeSkewedJoin` does run as part of query stage preparation as we previously discussed it's not needed. Before fine tuning the cost formula, I would appreciate if you could look at the rest of the changes and tell me if you still think moving `OptimizeSkewedJoin` to stage preparation is the right approach. It simplifies the logic in `AdaptiveSparkPlanExec` but complicates other places - compared to https://github.com/apache/spark/pull/30829 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
