[
https://issues.apache.org/jira/browse/SPARK-27812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16883716#comment-16883716
]
Stavros Kontopoulos edited comment on SPARK-27812 at 7/12/19 11:24 AM:
-----------------------------------------------------------------------
There is an issue in general with setting a driver
SparkUncaughtExceptionHandler as described in here:
[https://github.com/apache/spark/pull/24796]. I am also in favor of a handler,
where we can just sys.exit without running any shutdown hook logic though
because of the deadlock issue.
Btw I dont think we should downgrade we need to move forward and K8s moves fast
so we need to do the same. The upgrade happened because the client was very old
but jvm exception handling is a pita in general.
was (Author: skonto):
There is an issue in general with setting a driver
SparkUncaughtExceptionHandler as described in here:
[https://github.com/apache/spark/pull/24796]. I am also in favor of a handler,
where we can just sys.exit without running any shutdown hook logic though
because of the deadlock issue.
Btw I dont think we should downgrade we need to move forward and K8s moves fast
so we need to do the same. The upgrade happened because the client was very old.
> kubernetes client import non-daemon thread which block jvm exit.
> ----------------------------------------------------------------
>
> Key: SPARK-27812
> URL: https://issues.apache.org/jira/browse/SPARK-27812
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 2.4.3
> Reporter: Henry Yu
> Priority: Major
>
> I try spark-submit to k8s with cluster mode. Driver pod failed to exit with
> An Okhttp Websocket Non-Daemon Thread.
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
