skonto edited a comment on issue #24796: [SPARK-27900][CORE] Add uncaught exception handler to the driver URL: https://github.com/apache/spark/pull/24796#issuecomment-504558356 @squito I tried `XX:+ExitOnOutOfMemoryError` (described in jira) but the thing is executors do have proper uncaught exception handling. Driver does not have. So initially I tried to add something like that so things shutdown gracefully if possible given the jvm state but without the shutdown hooks. If shutdown hooks are enabled we have this issue with joins (which is reproducible 100%). Also, If you run the pi example on minikube and Spark will get stuck, so the initial issue where there are no uncaught exception handler is also 100% reproducible. Regarding the interrupts I havent seen it work so far but who knows maybe I got unlucky, the thing is it is also depends on the jdk version because there are fixes happening as described in my last comment, that change interrupts behavior. I called the openjdk guys to see if their fix relates to this, no response yet. I will try to to do the join in another thread and make sure it happens with higher priority, so far there is no guarantee for this.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
