Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21185#discussion_r184813348
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -299,6 +316,9 @@ private[spark] class Executor(
Thread.currentThread.setContextClassLoader(replClassLoader)
val ser = env.closureSerializer.newInstance()
logInfo(s"Running $taskName (TID $taskId)")
+ // When running in local mode, we might end up with the active
session from the driver set on
+ // this thread, though we never should, so we defensively clear it.
See SPARK-23894.
+ clearActiveSparkSessionMethod.foreach(_.invoke(null))
--- End diff --
Can this be done in the thread pool's thread factory instead?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
