Jacek Lewandowski created SPARK-8562:
----------------------------------------
Summary: Annoying messages about executor lost after stopping
SparkContext
Key: SPARK-8562
URL: https://issues.apache.org/jira/browse/SPARK-8562
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.3.1
Reporter: Jacek Lewandowski
Priority: Minor
This is a kind of a race condition because when the context is being stopped,
the executors are stopped first and then the actor system. However, sometimes
it happens that the actor system receives the information about stopping
executors before it gets shutdown and therefore it reports the "Executor lost"
error.
I think the simple way to fix that would be to log errors in
{{TaskSchedulerImpl.executorLost}} method, only if Spark context isn't stopped
(stopped flag is set to true first).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
