tgravescs commented on PR #36207: URL: https://github.com/apache/spark/pull/36207#issuecomment-1177748279
Note you have to take into account all running modes: cluster, client managed and unmanaged. If you are just thinking of client here, what about cluster mode. If the application is in cluster mode and it fails to unregistered and the Application master process dies, I would expect YARN to assume something bad happened and to rerun it. If you remove the staging directory that rerun is going to fail. That may not be ideal but it is better than the job showing up as failed to the user. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
