[
https://issues.apache.org/jira/browse/SPARK-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14487467#comment-14487467
]
Andrea Peruffo commented on SPARK-2949:
---------------------------------------
Which version of Spark?
ActorSystem has a method "registerOnTermination" that can trigger callbacks on
proper shutdown, by the way I've seen issues on netty shutdown under akka, the
only way I've found to live with it is to await polling until the os port
become free.
> SparkContext does not fate-share with ActorSystem
> -------------------------------------------------
>
> Key: SPARK-2949
> URL: https://issues.apache.org/jira/browse/SPARK-2949
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Reporter: Aaron Davidson
>
> It appears that an uncaught fatal error in Spark's Driver ActorSystem does
> not cause the SparkContext to terminate. We observed an issue in production
> that caused a PermGen error, but it just kept throwing this error:
> {code}
> 14/08/09 15:07:24 ERROR ActorSystemImpl: Uncaught fatal error from thread
> [spark-akka.actor.default-dispatcher-26] shutting down ActorSystem [spark]
> java.lang.OutOfMemoryError: PermGen space
> {code}
> We should probably do something similar for what we did in the DAGSCheduler
> and ensure that we call SparkContext#stop() if the entire ActorSystem dies
> with a fatal error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
