[ 
https://issues.apache.org/jira/browse/TOREE-355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Lavoillotte updated TOREE-355:
-------------------------------------
    Description: 
Using Toree 0.2.0.dev1 with CDH's embedded spark 2 did not cause TOREE-327 for 
me. Instead, I got this error:

{code}
Exception in thread "main" java.lang.IncompatibleClassChangeError: class 
org.clapper.classutil.asm.ASMEmptyVisitor has interface 
org.objectweb.asm.ClassVisitor as super class
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
        at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
        at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at org.clapper.classutil.asm.ClassFile$.load(ClassFinderImpl.scala:250)
        at 
org.clapper.classutil.ClassFinder.org$clapper$classutil$ClassFinder$$classData(ClassFinder.scala:427)
        at 
org.clapper.classutil.ClassFinder$$anonfun$2.apply(ClassFinder.scala:385)
        at 
org.clapper.classutil.ClassFinder$$anonfun$2.apply(ClassFinder.scala:385)
        at scala.collection.immutable.Stream.map(Stream.scala:418)
        at 
org.clapper.classutil.ClassFinder.processOpenZip(ClassFinder.scala:385)
        at org.clapper.classutil.ClassFinder.processJar(ClassFinder.scala:340)
        at 
org.clapper.classutil.ClassFinder.findClassesIn(ClassFinder.scala:329)
        at org.clapper.classutil.ClassFinder.find(ClassFinder.scala:320)
        at org.clapper.classutil.ClassFinder.getClasses(ClassFinder.scala:311)
        at 
org.apache.toree.plugins.PluginSearcher$$anonfun$1.apply(PluginSearcher.scala:73)
        at 
org.apache.toree.plugins.PluginSearcher$$anonfun$1.apply(PluginSearcher.scala:73)
        at scala.util.Try$.apply(Try.scala:192)
        at 
org.apache.toree.plugins.PluginSearcher.loadClassMap(PluginSearcher.scala:73)
        at 
org.apache.toree.plugins.PluginSearcher.internalClassInfo$lzycompute(PluginSearcher.scala:35)
        at 
org.apache.toree.plugins.PluginSearcher.internalClassInfo(PluginSearcher.scala:34)
        at 
org.apache.toree.plugins.PluginSearcher.internal$lzycompute(PluginSearcher.scala:38)
        at 
org.apache.toree.plugins.PluginSearcher.internal(PluginSearcher.scala:38)
        at 
org.apache.toree.plugins.PluginManager.internalPlugins$lzycompute(PluginManager.scala:45)
        at 
org.apache.toree.plugins.PluginManager.internalPlugins(PluginManager.scala:44)
        at 
org.apache.toree.plugins.PluginManager.initialize(PluginManager.scala:80)
        at 
org.apache.toree.boot.layer.StandardComponentInitialization$class.initializePlugins(ComponentInitialization.scala:221)
        at 
org.apache.toree.boot.layer.StandardComponentInitialization$class.initializeComponents(ComponentInitialization.scala:86)
        at org.apache.toree.Main$$anon$1.initializeComponents(Main.scala:35)
        at 
org.apache.toree.boot.KernelBootstrap.initialize(KernelBootstrap.scala:101)
        at 
org.apache.toree.Main$.delayedEndpoint$org$apache$toree$Main$1(Main.scala:40)
        at org.apache.toree.Main$delayedInit$body.apply(Main.scala:24)
{code}

Apparently in some much older version of ASM (3), {{ClassVisitor}} was an 
interface and not an abstract class. Said much older version can be found in 
some Hadoop libs or older Spark libs, and ends up in the classpath with CDH's 
spark 2.

The simplest solution is to shadow ASM, e.g. using JarJAr Links. The following 
produces an assembly jar that does not have this issue:

{code}
echo 'rule org.objectweb.asm.** org.apache.toree.shadow.@0' >relocate.rules
java -jar jarjar-1.4.jar process relocate.rules toree.jar toree-relocated.jar
{code}

ASM is a dependency of ClassUtil, for which [an issue on this is currently 
open|https://github.com/bmc/classutil/issues/19], but the author says he may 
not want to shadow ASM, so I thought maybe you might want to.

  was:
Using Toree 0.2.0.dev1 with CDH's embedded spark 2 did not cause TOREE-327 for 
me. Instead, I got this error:

{code}
Exception in thread "main" java.lang.IncompatibleClassChangeError: class 
org.clapper.classutil.asm.ASMEmptyVisitor has interface 
org.objectweb.asm.ClassVisitor as super class
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
        at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
        at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at org.clapper.classutil.asm.ClassFile$.load(ClassFinderImpl.scala:250)
        at 
org.clapper.classutil.ClassFinder.org$clapper$classutil$ClassFinder$$classData(ClassFinder.scala:427)
        at 
org.clapper.classutil.ClassFinder$$anonfun$2.apply(ClassFinder.scala:385)
        at 
org.clapper.classutil.ClassFinder$$anonfun$2.apply(ClassFinder.scala:385)
        at scala.collection.immutable.Stream.map(Stream.scala:418)
        at 
org.clapper.classutil.ClassFinder.processOpenZip(ClassFinder.scala:385)
        at org.clapper.classutil.ClassFinder.processJar(ClassFinder.scala:340)
        at 
org.clapper.classutil.ClassFinder.findClassesIn(ClassFinder.scala:329)
        at org.clapper.classutil.ClassFinder.find(ClassFinder.scala:320)
        at org.clapper.classutil.ClassFinder.getClasses(ClassFinder.scala:311)
        at 
org.apache.toree.plugins.PluginSearcher$$anonfun$1.apply(PluginSearcher.scala:73)
        at 
org.apache.toree.plugins.PluginSearcher$$anonfun$1.apply(PluginSearcher.scala:73)
        at scala.util.Try$.apply(Try.scala:192)
        at 
org.apache.toree.plugins.PluginSearcher.loadClassMap(PluginSearcher.scala:73)
        at 
org.apache.toree.plugins.PluginSearcher.internalClassInfo$lzycompute(PluginSearcher.scala:35)
        at 
org.apache.toree.plugins.PluginSearcher.internalClassInfo(PluginSearcher.scala:34)
        at 
org.apache.toree.plugins.PluginSearcher.internal$lzycompute(PluginSearcher.scala:38)
        at 
org.apache.toree.plugins.PluginSearcher.internal(PluginSearcher.scala:38)
        at 
org.apache.toree.plugins.PluginManager.internalPlugins$lzycompute(PluginManager.scala:45)
        at 
org.apache.toree.plugins.PluginManager.internalPlugins(PluginManager.scala:44)
        at 
org.apache.toree.plugins.PluginManager.initialize(PluginManager.scala:80)
        at 
org.apache.toree.boot.layer.StandardComponentInitialization$class.initializePlugins(ComponentInitialization.scala:221)
        at 
org.apache.toree.boot.layer.StandardComponentInitialization$class.initializeComponents(ComponentInitialization.scala:86)
        at org.apache.toree.Main$$anon$1.initializeComponents(Main.scala:35)
        at 
org.apache.toree.boot.KernelBootstrap.initialize(KernelBootstrap.scala:101)
        at 
org.apache.toree.Main$.delayedEndpoint$org$apache$toree$Main$1(Main.scala:40)
        at org.apache.toree.Main$delayedInit$body.apply(Main.scala:24)
{code}

Apparently in some much older version of ASM (3), {{ClassVisitor}} was an 
interface and not an abstract class. Said much older version can be found in 
some Hadoop libs or older Spark libs, and ends up in the classpath with CDH's 
spark 2.

The simplest solution is to shadow ASM, e.g. using JarJAr Links. The following 
produces an assembly jar that does not have this issue:

{code}
echo 'rule org.objectweb.asm.** org.apache.toree.shadow.@0' >relocate.rules
java -jar jarjar-1.4.jar process relocate.rules toree.jar toree-relocated.jar
{code}

ASM is a dependency of ClassUtil, for which [an issue on this is currently 
open|https://github.com/bmc/classutil/issues/19], but the author says he may 
not want to shadow ASM, so I thought maybe you guy would want to.


> java.lang.IncompatibleClassChangeError: class 
> org.clapper.classutil.asm.ASMEmptyVisitor has interface 
> org.objectweb.asm.ClassVisitor as super class
> ---------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: TOREE-355
>                 URL: https://issues.apache.org/jira/browse/TOREE-355
>             Project: TOREE
>          Issue Type: Bug
>         Environment: Toree 0.2.0.dev1
> CDH 5.9's embedded spark-2.0.0-beta2
>            Reporter: Adrien Lavoillotte
>              Labels: easyfix
>
> Using Toree 0.2.0.dev1 with CDH's embedded spark 2 did not cause TOREE-327 
> for me. Instead, I got this error:
> {code}
> Exception in thread "main" java.lang.IncompatibleClassChangeError: class 
> org.clapper.classutil.asm.ASMEmptyVisitor has interface 
> org.objectweb.asm.ClassVisitor as super class
>         at java.lang.ClassLoader.defineClass1(Native Method)
>         at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
>         at 
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>         at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
>         at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>         at java.lang.ClassLoader.defineClass1(Native Method)
>         at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
>         at 
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>         at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
>         at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>         at 
> org.clapper.classutil.asm.ClassFile$.load(ClassFinderImpl.scala:250)
>         at 
> org.clapper.classutil.ClassFinder.org$clapper$classutil$ClassFinder$$classData(ClassFinder.scala:427)
>         at 
> org.clapper.classutil.ClassFinder$$anonfun$2.apply(ClassFinder.scala:385)
>         at 
> org.clapper.classutil.ClassFinder$$anonfun$2.apply(ClassFinder.scala:385)
>         at scala.collection.immutable.Stream.map(Stream.scala:418)
>         at 
> org.clapper.classutil.ClassFinder.processOpenZip(ClassFinder.scala:385)
>         at org.clapper.classutil.ClassFinder.processJar(ClassFinder.scala:340)
>         at 
> org.clapper.classutil.ClassFinder.findClassesIn(ClassFinder.scala:329)
>         at org.clapper.classutil.ClassFinder.find(ClassFinder.scala:320)
>         at org.clapper.classutil.ClassFinder.getClasses(ClassFinder.scala:311)
>         at 
> org.apache.toree.plugins.PluginSearcher$$anonfun$1.apply(PluginSearcher.scala:73)
>         at 
> org.apache.toree.plugins.PluginSearcher$$anonfun$1.apply(PluginSearcher.scala:73)
>         at scala.util.Try$.apply(Try.scala:192)
>         at 
> org.apache.toree.plugins.PluginSearcher.loadClassMap(PluginSearcher.scala:73)
>         at 
> org.apache.toree.plugins.PluginSearcher.internalClassInfo$lzycompute(PluginSearcher.scala:35)
>         at 
> org.apache.toree.plugins.PluginSearcher.internalClassInfo(PluginSearcher.scala:34)
>         at 
> org.apache.toree.plugins.PluginSearcher.internal$lzycompute(PluginSearcher.scala:38)
>         at 
> org.apache.toree.plugins.PluginSearcher.internal(PluginSearcher.scala:38)
>         at 
> org.apache.toree.plugins.PluginManager.internalPlugins$lzycompute(PluginManager.scala:45)
>         at 
> org.apache.toree.plugins.PluginManager.internalPlugins(PluginManager.scala:44)
>         at 
> org.apache.toree.plugins.PluginManager.initialize(PluginManager.scala:80)
>         at 
> org.apache.toree.boot.layer.StandardComponentInitialization$class.initializePlugins(ComponentInitialization.scala:221)
>         at 
> org.apache.toree.boot.layer.StandardComponentInitialization$class.initializeComponents(ComponentInitialization.scala:86)
>         at org.apache.toree.Main$$anon$1.initializeComponents(Main.scala:35)
>         at 
> org.apache.toree.boot.KernelBootstrap.initialize(KernelBootstrap.scala:101)
>         at 
> org.apache.toree.Main$.delayedEndpoint$org$apache$toree$Main$1(Main.scala:40)
>         at org.apache.toree.Main$delayedInit$body.apply(Main.scala:24)
> {code}
> Apparently in some much older version of ASM (3), {{ClassVisitor}} was an 
> interface and not an abstract class. Said much older version can be found in 
> some Hadoop libs or older Spark libs, and ends up in the classpath with CDH's 
> spark 2.
> The simplest solution is to shadow ASM, e.g. using JarJAr Links. The 
> following produces an assembly jar that does not have this issue:
> {code}
> echo 'rule org.objectweb.asm.** org.apache.toree.shadow.@0' >relocate.rules
> java -jar jarjar-1.4.jar process relocate.rules toree.jar toree-relocated.jar
> {code}
> ASM is a dependency of ClassUtil, for which [an issue on this is currently 
> open|https://github.com/bmc/classutil/issues/19], but the author says he may 
> not want to shadow ASM, so I thought maybe you might want to.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to