tgravescs commented on issue #22173: [SPARK-24355] Spark external shuffle server improvement to better handle block fetch requests. URL: https://github.com/apache/spark/pull/22173#issuecomment-573105558 the jobs we ran didn't see an overall performance impact and it greatly helped when you ran on busy multi-tenant cluster. How about if the value of spark.shuffle.server.chunkFetchHandlerThreadsPercent isn't explicitly set by the user we just don't add the extra event loop or thread pool, see TransportContext - just like it does it its not shuffle, that should be a fairly easy change.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
