GitHub user jiangxb1987 opened a pull request:
https://github.com/apache/spark/pull/18614
[SPARK-18646][REPL] Set parent classloader as null for ExecutorClassLoader
## What changes were proposed in this pull request?
`ClassLoader` will preferentially load class from `parent`. Only when
`parent` is null or the load failed, that it will call the overridden
`findClass` function. To avoid the potential issue caused by loading class
using inappropriate class loader, we should set the `parent` of `ClassLoader`
to null, so that we can fully control which class loader is used.
This is take over of #17074, the primary author of this PR is @taroplus .
## How was this patch tested?
Add test case in `ExecutorClassLoaderSuite`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jiangxb1987/spark executor_classloader
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18614.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18614
----
commit e0d95381ffc70c22bc2c8a27f908d97f565b2566
Author: Kohki Nishio <[email protected]>
Date: 2017-02-26T16:22:03Z
Set parent classloader as null for ExecutorClassLoader
This change is to address a classloader problem under sbt environment
more details can be found in Jira
[https://issues.apache.org/jira/browse/SPARK-19675]
commit 0a490bbb48803a72b8dd5b3e7ccd4336bf1e4c46
Author: Kohki Nishio <[email protected]>
Date: 2017-02-26T18:05:51Z
Add a UT for ExecutorClassLoader
commit 13b016747c6f70584268a6a06c76c2211cf2c9bd
Author: Kohki Nishio <[email protected]>
Date: 2017-02-26T18:10:09Z
Fixed unwanted import change
commit d66ad0cbd30ed4f834d62e559456dd4bb03b747e
Author: Kohki Nishio <[email protected]>
Date: 2017-03-03T02:48:50Z
Add comment for disabling scalastyle
commit 92c810f67591f12a0ea71aae7d8cc19b54d180d4
Author: Kohki Nishio <[email protected]>
Date: 2017-03-03T03:03:15Z
Another attempt to resolve scala style issue
commit f9204c2b4efd5e11429262772f379738f2ce2cbd
Author: Kohki Nishio <[email protected]>
Date: 2017-03-03T03:45:43Z
ScalaStyle complains about comment
commit f9f97706f1508bfba8d00b8727af259868c88c2c
Author: Kohki Nishio <[email protected]>
Date: 2017-06-21T05:05:50Z
Add a comment to explain why this class loader does not set parent
classloader
commit d26f89f9ba1008a247d9332f3c7c34a13cd55ce8
Author: Xingbo Jiang <[email protected]>
Date: 2017-07-12T15:35:34Z
update comments
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
