GitHub user sancyx opened a pull request:
https://github.com/apache/zeppelin/pull/2771
ZEPPELIN-3209. Preserve thread context classloader when running jobs in
RemoteInterpreterServer
### What is this PR for?
Spark jobs may change current thread context classloader sometimes.
For example in case of issue ZEPPELIN-2475 using Spark 2.2 and Scala 2.11.8
when you run DepInterpreter that will start SparkILoop --> ILoop which changes
the current thread context classloader, from LauncherAppClassloader to
ScalaClassloader@URLClassloader. This result in classloading problems when
SparkInterpreter is trying to build up Spark Context, in case SparkInterpreter
is started by scheduler on same thread.
In short when running subsequent paragraphs, users will get an ambiguous
NullPointerException, which is hard to understand as it hides the root cause of
the problem. As a safety measure to prevent such cases RemoteInterpreterServer
should save & restore original thread context classloader.
### What type of PR is it?
[Bug Fix]
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3209
### How should this be tested?
* Use Spark 2.2 and Scala 2.11.8
* create a notebook with two paragraphs
* in first paragraph use %spark.dep interpreter and add load some
dependencies
* in second paragraph use %spark interpreter and run some spark code
### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/banzaicloud/zeppelin master-ZEPPELIN-3209
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2771.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2771
----
commit 0411745566ccc4427e9da407668a83bb922ab667
Author: Magyari Sandor Szilard <sancyx@...>
Date: 2018-01-31T17:10:18Z
ZEPPELIN-3209. Preserve thread context classloader when running jobs in
RemoteInterpreterServer (magyari_sandor)
----
---
