[GitHub] [spark] mridulm commented on pull request #37779: [wip][SPARK-40320][Core] Executor should exit when it failed to initialize for fatal error

2022-09-27 Thread GitBox
mridulm commented on PR #37779: URL: https://github.com/apache/spark/pull/37779#issuecomment-1259029762 Can you take a look at comment above @yabola and work on the fix ? Since you already spent a lot of time on this. -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] mridulm commented on pull request #37779: [wip][SPARK-40320][Core] Executor should exit when it failed to initialize for fatal error

2022-09-26 Thread GitBox
mridulm commented on PR #37779: URL: https://github.com/apache/spark/pull/37779#issuecomment-1259007734 Thanks for the query @Ngone51 - I missed out one aspect of my analysis, which ends up completely changing the solution - my bad :-( The answer to your query has the reason for the

[GitHub] [spark] mridulm commented on pull request #37779: [wip][SPARK-40320][Core] Executor should exit when it failed to initialize for fatal error

2022-09-26 Thread GitBox
mridulm commented on PR #37779: URL: https://github.com/apache/spark/pull/37779#issuecomment-1258716473 Added a few debug statements, and it became clear what the issue is. Essentially, since we are leveraging a `ThreadPoolExecutor`, it does not result in killing the thread with the

[GitHub] [spark] mridulm commented on pull request #37779: [wip][SPARK-40320][Core] Executor should exit when it failed to initialize for fatal error

2022-09-26 Thread GitBox
mridulm commented on PR #37779: URL: https://github.com/apache/spark/pull/37779#issuecomment-1258427841 I agree @Ngone51, it should have :-) I am trying to reproduce this locally and see if we are missing something - or there is some nuance behind the observed behavior. -- This is an

[GitHub] [spark] mridulm commented on pull request #37779: [wip][SPARK-40320][Core] Executor should exit when it failed to initialize for fatal error

2022-09-23 Thread GitBox
mridulm commented on PR #37779: URL: https://github.com/apache/spark/pull/37779#issuecomment-1255831313 I will try to reproduce this locally - is there anything else specific I need to know to do so ? Or is the instructions in SPARK-40320 sufficient ? Thx -- This is an automated message

[GitHub] [spark] mridulm commented on pull request #37779: [wip][SPARK-40320][Core] Executor should exit when it failed to initialize for fatal error

2022-09-22 Thread GitBox
mridulm commented on PR #37779: URL: https://github.com/apache/spark/pull/37779#issuecomment-1255648587 A few points: * Plugins are initialized as part of `Executor` construction, but after uncaught exception handler is set. * If anything other than `NonFatal` is thrown in

[GitHub] [spark] mridulm commented on pull request #37779: [wip][SPARK-40320][Core] Executor should exit when it failed to initialize for fatal error

2022-09-21 Thread GitBox
mridulm commented on PR #37779: URL: https://github.com/apache/spark/pull/37779#issuecomment-1254457075 I am not sure I follow what the code snippet is trying to do. Changing the code to : ``` private def receiveLoop() {