pan3793 commented on PR #46889: URL: https://github.com/apache/spark/pull/46889#issuecomment-2151326847
This patch covers the SPARK-34674 and SPARK-42698 cases. And maybe the code introduced in SPARK-34674 can be removed. SPARK-34674 is used to tackle the Spark on K8s exit issues, it also introduces side-effects: - the shutdown process is different between K8s and other platforms - K8s shutdowns SparkContext ahead, and relies on JVM shutdown hooks to handle other shutdown logic - as SPARK-42219 pointed out, SparkContext always shutdown after the Main method has finished. In SPARK-42698, a similar shutdown logic was requested for the YARN client mode to propagate the exit code to AM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
