[GitHub] [spark] dongjoon-hyun commented on pull request #40128: [SPARK-42466][K8S]: Cleanup k8s upload directory when job terminates

2023-06-06 Thread via GitHub
dongjoon-hyun commented on PR #40128: URL: https://github.com/apache/spark/pull/40128#issuecomment-1579579865 Sorry but I'm still in the same position, @shrprasa . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [spark] dongjoon-hyun commented on pull request #40128: [SPARK-42466][K8S]: Cleanup k8s upload directory when job terminates

2023-03-22 Thread via GitHub
dongjoon-hyun commented on PR #40128: URL: https://github.com/apache/spark/pull/40128#issuecomment-1480402673 @shrprasa . 1. It seems that you have an assumption that Shutdown hook is magically reliable. However, shutdown hook has a well-known limitation where JVM can be destroyed a