Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22192#discussion_r216049113
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -219,6 +236,13 @@ private[spark] class Executor(
heartbeater.shutdown()
heartbeater.awaitTermination(10, TimeUnit.SECONDS)
threadPool.shutdown()
+
+ // Notify plugins that executor is shutting down so they can terminate
cleanly
+ // oldContextClassLoader is redefined in case the class loader changed
during execution
+ val oldContextClassLoader = Utils.getContextOrSparkClassLoader
+ Thread.currentThread.setContextClassLoader(replClassLoader)
+ plugins.foreach(_.shutdown())
--- End diff --
Just so Mridul's comment doesn't get lost: you need to `try` around the
shutdown call.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
