Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/3233#discussion_r23413919
--- Diff:
core/src/main/scala/org/apache/spark/executor/ExecutorURLClassLoader.scala ---
@@ -32,36 +35,40 @@ private[spark] trait MutableURLClassLoader extends
ClassLoader {
}
private[spark] class ChildExecutorURLClassLoader(urls: Array[URL], parent:
ClassLoader)
- extends MutableURLClassLoader {
-
- private object userClassLoader extends URLClassLoader(urls, null){
- override def addURL(url: URL) {
- super.addURL(url)
- }
- override def findClass(name: String): Class[_] = {
- super.findClass(name)
- }
- }
+ extends URLClassLoader(urls, null) with MutableURLClassLoader {
private val parentClassLoader = new ParentClassLoader(parent)
- override def findClass(name: String): Class[_] = {
+ override def loadClass(name: String, resolve: Boolean): Class[_] = {
try {
- userClassLoader.findClass(name)
--- End diff --
What do you mean by "cache"? There's no reference to "cache" anywhere in
the documentation of `ClassLoader`. `loadClass` is only overridden so that the
normal lookup order is inverted - normally it would be `parent, this` and now
it's `this, parent`. If there's any caching done anywhere, I'd expect the
actual class loader implementation (`super` or `parentClassLoader` in this
code) to handle that.
As for locking, the comments from the JDK 1.7 docs seems to indicate the
Spark use case is safe (it mentions deadlocks in the cases of non-hierarchical
classloader, which I don't think applies here). But, if that's a concern,
adding per-name locking shouldn't be hard (but could be pretty expensive
depending on how it's done).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
