srowen commented on issue #23401: [SPARK-26513][Core] : Trigger GC on executor node idle URL: https://github.com/apache/spark/pull/23401#issuecomment-451962870 It would require more info from the driver, and even the driver isn't entirely sure when the next tasks will schedule. I think Ryan's point stands that it's hard to know how much a GC will save. You don't want to keep triggering a GC that isn't going to free memory, but we won't know whether there is garbage or not. Then we get into defining a waiting period between GCs or something, and have started reimplementing GC. I think we would have to close this until there is better evidence that this can be implemented cleanly and for a perf gain.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
