igreenfield commented on issue #1931: [SPARK-3015] Block on cleaning tasks to prevent Akka timeouts URL: https://github.com/apache/spark/pull/1931#issuecomment-483139476 @andrewor14 I see in the code: /** * Whether the cleaning thread will block on cleanup tasks (other than shuffle, which * is controlled by the `spark.cleaner.referenceTracking.blocking.shuffle` parameter). * * Due to SPARK-3015, this is set to true by default. This is intended to be only a temporary * workaround for the issue, which is ultimately caused by the way the BlockManager endpoints * issue inter-dependent blocking RPC messages to each other at high frequencies. This happens, * for instance, when the driver performs a GC and cleans up all broadcast blocks that are no * longer in scope. */ does that still needed?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
