Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/21923#discussion_r207036143
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -130,6 +130,12 @@ private[spark] class Executor(
private val urlClassLoader = createClassLoader()
private val replClassLoader = addReplClassLoaderIfNeeded(urlClassLoader)
+ Thread.currentThread().setContextClassLoader(replClassLoader)
--- End diff --
My memory monitor would be fine if the constructor were called in another
thread. (It actually creates its own thread -- it has to, as its going to
continually poll.)
What would be the advantage to calling the constructor in a separate
thread? If its just protect against exceptions, we could just do a try/catch.
If its to ensure that we don't tie up the main executor threads ... well, even
in another thread, the plugin could do something arbitrary to tie up all the
resources associated with this executor (eg. launch 30 threads and do something
intensive in each one).
Not opposed to having another thread, just want to understand why.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
