Hi, I have a question that is probably related to SPARK-1870 <https://issues.apache.org/jira/browse/SPARK-1870>. Basically I have also encountered the issue that with separate classloaders while developing a programming framework where I have to use reflection inside the application code.
To simply the question, let's say my framework code depends on a jar with an older version and the application code depends on the same jar with a newer version. And let's say that jar has a class named CommonClass. I used a customized post-delegation class loader so that these two version can exist separately into separate class loader. And if I do sth. like: public static void main(String arg[]) throws Exception { // Say this is framework code new CommonClass().printJarVersion(); // Set the customized class loader as thread context class loader Thread thread = Thread.currentThread(); ClassLoader oldClassLoader = thread.getContextClassLoader(); File appJar = new File("/dir/to/app/class/path"); URL[] classpath = new URL[] { appJar.toURI().toURL() }; PostDelegationClassLoader newClassLoader = new PostDelegationClassLoader(classpath); thread.setContextClassLoader(newClassLoader); try { // Say this is application code, like process(). new CommonClass().printJarVersion(); thread.getContextClassLoader().loadClass("CommonClass").newInstance().printJarVersion(); } finally { thread.setContextClassLoader(oldClassLoader); } } ----------------------- It will print: CommonClass: version 1 CommonClass: version 1 CommonClass: version 2 As one can see I have to use reflection to explicitly specify the customized class loader if I want to create the class with the new version. This is definitely bad for the users. I saw there are some discussions around this issue on SPARK-1870, but not so sure what is the final solution to this. Anyone could help me out? -- Guozhang