skonto commented on issue #24796: [SPARK-27900][CORE] Add uncaught exception handler to the driver URL: https://github.com/apache/spark/pull/24796#issuecomment-504558356 @squito I tried it `XX:+ExitOnOutOfMemoryError` (described in jira) but the thing is executors do have proper uncaught exception handling. Driver does not have. So initially I tried to add something like that so things shutdown gracefully if possible given the jvm state but without the shutdown hooks. If shutdown hooks are enabled we have this issue with joins (which is reproducible 100%). Also, If you run the pi example on minikube and Spark will get stuck, so the initial issue where there are no uncaught exception handler is also 100% reproducible.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
