peter-toth edited a comment on issue #26044: [SPARK-29375][SQL] Exchange reuse across all subquery levels URL: https://github.com/apache/spark/pull/26044#issuecomment-552225930 > Could you show us actual performance numbers with/without this pr? I created a simple benchmark here: https://github.com/peter-toth/spark/commit/06a7f0ad1f647d7e23d2fc9365801c2ae888c5ef just to show that this PR (`originalReuseExchangeVersion - false`) makes sense in some cases. ``` Running benchmark: Exchange reuse 1 Running case: originalReuseExchangeVersion - true Stopped after 2 iterations, 24415 ms Running case: originalReuseExchangeVersion - false Stopped after 2 iterations, 13158 ms OpenJDK 64-Bit Server VM 1.8.0_212-b04 on Mac OS X 10.14.6 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz Exchange reuse 1: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ originalReuseExchangeVersion - true 11946 12208 370 0.0 2389167936.6 1.0X originalReuseExchangeVersion - false 6538 6579 59 0.0 1307541708.0 1.8X Running benchmark: Exchange reuse 2 Running case: originalReuseExchangeVersion - true Stopped after 2 iterations, 23746 ms Running case: originalReuseExchangeVersion - false Stopped after 2 iterations, 13400 ms OpenJDK 64-Bit Server VM 1.8.0_212-b04 on Mac OS X 10.14.6 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz Exchange reuse 2: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ originalReuseExchangeVersion - true 11760 11873 160 0.0 2352022663.0 1.0X originalReuseExchangeVersion - false 6642 6700 82 0.0 1328495627.4 1.8X ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
