[GitHub] [spark] AngersZhuuuu commented on pull request #40314: [SPARK-42698][CORE] SparkSubmit should also stop SparkContext when exit program in yarn mode and pass exitCode to AM side

AngersZh commented on PR #40314: URL: https://github.com/apache/spark/pull/40314#issuecomment-1500073974 ping @dongjoon-hyun @HyukjinKwon @attilapiros @srowen -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [spark] AngersZhuuuu commented on pull request #40314: [SPARK-42698][CORE] SparkSubmit should also stop SparkContext when exit program in yarn mode and pass exitCode to AM side

AngersZh commented on PR #40314: URL: https://github.com/apache/spark/pull/40314#issuecomment-1465695376 > @AngersZh can you comment on the original discussion thread and convince related people to add back `sc.stop`? [#32081 (comment)](https://github.com/apache/spark/pull/32081#dis

[GitHub] [spark] AngersZhuuuu commented on pull request #40314: [SPARK-42698][CORE] SparkSubmit should also stop SparkContext when exit program in yarn mode and pass exitCode to AM side

AngersZh commented on PR #40314: URL: https://github.com/apache/spark/pull/40314#issuecomment-1463391491 @cloud-fan Seems this code https://github.com/apache/spark/pull/32283 first want to fix issue in k8s, then @dongjoon-hyun make it limit in k8s env. But this also can work for yarn en

[GitHub] [spark] AngersZhuuuu commented on pull request #40314: [SPARK-42698][CORE] SparkSubmit should also stop SparkContext when exit program in yarn mode and pass exitCode to AM side

AngersZh commented on PR #40314: URL: https://github.com/apache/spark/pull/40314#issuecomment-1463389076 Failed UT should not related to this pr. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AngersZhuuuu commented on pull request #40314: [SPARK-42698][CORE] SparkSubmit should also stop SparkContext when exit program in yarn mode and pass exitCode to AM side

AngersZh commented on PR #40314: URL: https://github.com/apache/spark/pull/40314#issuecomment-1463340552 > This seems to be a revert of #33403 as now we stop SparkContext in YARN environment as well. We should justify it in the PR description. This is not simply passing the exitCode. Pl