Thomas Graves created SPARK-14405:
-------------------------------------
Summary: broadcast variables can cause lots of warning messages on
shutdown
Key: SPARK-14405
URL: https://issues.apache.org/jira/browse/SPARK-14405
Project: Spark
Issue Type: Bug
Components: Spark Core
Reporter: Thomas Graves
when using broadcast variables and lots of executors you get a lot of warning
messages on shutdown.
Messages from LiveListenerBus are like:
SparkListenerBus has already stopped! Dropping event .....
The issue here is the order of the stop. In SparkContext.stop, ListenerBus.stop
is earlier than SparkEnv.stop is called and thus it gets shutdown before the
broadcast manager or with https://github.com/apache/spark/pull/12113 the
MapOutputTracker.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
