peter-toth commented on pull request #28885: URL: https://github.com/apache/spark/pull/28885#issuecomment-776590219
@cloud-fan could you please help me with this PR? The incorrect reuse nodes cause performance issues currently and in special cases they can even cause performance regression when someone upgrades from 2.x to 3.x. An example is TPCDS q23b in which the second traversal in `ReuseExchange` rule (added with DPP in Spark 3) ruins a better, bigger reuse node. I see that AQE is the main way now, where these issues doesn't come up, but I think we are still far from full AQE support and we should fix the non-adaptive path as well. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
