ahshahid commented on PR #37824: URL: https://github.com/apache/spark/pull/37824#issuecomment-1242223041
> Agree with @peter-toth that we should do all ordering in the 2nd pass, in a bottom-up way. I suppose @cloud-fan @peter-toth you want to code the change...? Or you want me to generate a new PR .. I think that making it bottom-up would result in many tree traversals for commutative expressions resulting in perf degradation .. The reason I think is this: In your current code of top to bottom, the first encountered commutative expression will flatten all the consecutive similar commutative expressions ,so only one such recurring traversal will be needed. But if you go from bottom to top, at the re-order will occur at each child commutative expression... May be I can test the recommended change in a test & gauge the perf.. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
