srowen commented on issue #24796: [SPARK-27900][CORE] Add uncaught exception handler to the driver URL: https://github.com/apache/spark/pull/24796#issuecomment-499106393 I think the test failure is related somehow as it concerns how the app exits; not exactly sure yet. I think removing all the hooks is too heavy a hammer. It seems like `EventLoop.stop()` blocking on the event thread is problematic here. ``` def stop(): Unit = { if (stopped.compareAndSet(false, true)) { eventThread.interrupt() var onStopCalled = false try { eventThread.join() // Call onStop after the event thread exits to make sure onReceive happens before onStop ``` @zsxwing is this `join()` really necessary? I see the comment here. But here, an OOM exception crashes the thread, it invokes the shutdown hooks which stop the `EventLoop`, but then it's waiting on itself to stop. If this is removed, I am not sure what goes wrong. If `onStop` happens before or after an `onReceive` during shutdown, that's fine. If it happens during `onReceive`, well, it has already been interrupted and won't complete normally.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
