Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/22246#discussion_r213429435
--- Diff: repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala ---
@@ -43,10 +44,26 @@ class SparkILoop(in0: Option[BufferedReader], out:
JPrintWriter)
def this(in0: BufferedReader, out: JPrintWriter) = this(Some(in0), out)
def this() = this(None, new JPrintWriter(Console.out, true))
+ // TODO: Remove the following `override` when the support of Scala 2.11
is ended
+ // Scala 2.11 has a bug of finding imported types in class constructors,
extends clause
+ // which is fixed in Scala 2.12 but never be back-ported into Scala
2.11.x.
+ // As a result, we copied the fixes into `SparkILoopInterpreter`. See
SPARK-22393 for detail.
override def createInterpreter(): Unit = {
- intp = new SparkILoopInterpreter(settings, out)
+ if (isScala2_11) {
+ if (addedClasspath != "") {
+ settings.classpath append addedClasspath
+ }
+ intp =
Utils.classForNameFromSpark("org.apache.spark.repl.SparkILoopInterpreter")
+ .getDeclaredConstructor(Seq(classOf[Settings],
classOf[JPrintWriter]): _*)
--- End diff --
To summarize the classLoader issue here, `classOf[Settings]` will use the
classloader of the object of `SparkILoop`, and `Utils.classForName` will use
the classloader of the current running thread which can lead to mismatching.
I think using `Utils.classForNameFromSpark` is dangerous since there is no
guaranteed that this will use the classloader of this object which is used in
`classOf[Settings]`.
I will use `Class.ofName` to ensure the consistency.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
