SandishKumarHN commented on issue #23401: [SPARK-26513][Core] : Trigger GC on executor node idle URL: https://github.com/apache/spark/pull/23401#issuecomment-450699195 @srowen when we used -XX: InitiatingHeapOccupancyPercent=~25 we didn't see much difference in performance but there were frequent minor GC's and I forgot to mention we only use these settings for large batch and long-running job's. for normal or jobs which finishes within a few minutes or a couple hours, we did not use these settings. because we did not see any performance improvements for those jobs. and we use static allocation. I would say this will be helpful for long-running job's only. 1) Why would Empty Executor have heap usage > .8? 2) Full GC will help when heap percentage usage more than >.8 with empty running tasks. I agree with points 3, 4, 5.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
