fitermay edited a comment on issue #23986: [SPARK-27070] Fix performance bug in DefaultPartitionCoalescer URL: https://github.com/apache/spark/pull/23986#issuecomment-470918335 @fitermay @attilapiros I've pushed the benchmark and the suggestion of using filter, map. From the benchmark it's seems that host=1 is the absolute worst case and this change improves it by a decent margin. It improves the other cases slightly.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
