> On 3 Feb 2016, at 08:07, Chen, Haifeng <haifeng.c...@intel.com> wrote: > >>> [Colin] I do think there will be some challenges splitting this >>> functionality out into a separate jar, because of the way our CLASSPATH >>> works right now. > Yes, this challenges are common for shared libraries in Java. Just as you > mentioned, keeping API compatibility or using classpath isolation are two > practical methods.
you can't isolate JNI. This is why NMs are moving to to a separate JVM for plugins - the Spark 1.6 shuffle is incompatible with Hadoop (currently: jackson)
