jiangxb1987 commented on issue #23794: [SPARK-26884][CORE] Let task acquire memory accurately when using spilled memory URL: https://github.com/apache/spark/pull/23794#issuecomment-464774721 Please note that we want to let each task get at least 1 / (2 * numActiveTasks) of the total memory pool, if we can't offer that much then we INTENTIONALLY wait for other tasks to free up more memories. Thus, I'm not sure the change proposed by this PR is the direction we want to follow.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
