Github user sryza commented on a diff in the pull request:
https://github.com/apache/spark/pull/9313#discussion_r43682628
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -175,6 +175,29 @@ private[spark] object Utils extends Logging {
}
/**
+ * Update the current threads class loader.
+ * Requires the current class loader is a MutableURLClassLoader,
otherwise skips updating with a
+ * warning. Intended for use by addJar(), although constructing an
instance of the class will
+ * still require:
+ *
sc._jvm.java.lang.Thread.currentThread().getContextClassLoader().loadClass("class
name")
+ * as described in SPARK-5185.
+ */
+ def updatePrimaryClassLoader(sc: SparkContext) {
--- End diff --
What's the advantage of this update method that compares the full contents
of `sc.addedJars` to those already registered with the current classloader, vs.
just calling `cs.addUrl` directly on the jar being added?
A possibly confusing effect of this approach is that if I have code like:
sc.addJar(jar1)
... other stuff blah blah blah ...
sc.addJar(jar2, true)
jar1 will get added to the classpath only when `sc.addJar` is called the
second time.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
