hdaikoku commented on PR #42426: URL: https://github.com/apache/spark/pull/42426#issuecomment-1727835491
> I think `SparkUncaughtExceptionHandler` should caught this OOM exception and abort the executor. I'm not sure if I'm following this. For this particular case, OOM was actually caught at [`AbstractChannelHandlerContext.invokeExceptionCaught`](https://github.com/netty/netty/blob/netty-4.1.74.Final/transport/src/main/java/io/netty/channel/AbstractChannelHandlerContext.java#L304). So it was not an uncaught exception. Also, if I understand correctly from the previous discussion in https://github.com/apache/spark/pull/37779, the problem of `submit()` is that it swallows uncaught exceptions that have been thrown from _inside_ a spawned thread, while, here, the OOM was thrown within `submit()` itself even before a thread was spawned. > ``` > at java.lang.Thread.start0(Native Method) > at java.lang.Thread.start(Thread.java:719) > at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957) > at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378) > at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) > ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
