Hi, This is pretty hard to say what is the root cause, especially in complex deployments like CDH. Most probably you JAR is packaged incorrectly because your application is able to load Ignite classes, but cannot load jcache API. Could you try to simply put cache-api-1.0.0.jar to all places and scripts where you added Igntte JARs and/or your jar-with-dependencies?
Vladimir. On Tue, Apr 26, 2016 at 12:43 AM, mdolgonos <[email protected]> wrote: > Vladimir, > There are 2 things that I'm experiencing so far: > 1. I have added the following code to spark-env.sh in my CDH installation > IGNITE_HOME=/etc/ignite-1.5.0 > IGNITE_LIBS="${IGNITE_HOME}/libs/*" > > for file in ${IGNITE_HOME}/libs/* > do > if [ -d ${file} ] && [ "${file}" != "${IGNITE_HOME}"/libs/optional ]; > then > IGNITE_LIBS=${IGNITE_LIBS}:${file}/* > fi > done > > but Ignite jars are still not recognized by Spark after restarting Spark as > well as the entire CDH > . My location of CDH is the default one: > /opt/cloudera/parcels/CDH-5.5.2-1.cdh5.5.2.p0.4/etc/spark/conf.dist > So I decided to compile my code into a jar-with-dependencies which led me > to > the second issue: > > 2. Looks like the jars were discovered by spark-submit, however, now I'm > getting the following exception: > Exception in thread "main" java.lang.NoClassDefFoundError: > javax/cache/configuration/MutableConfiguration > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:800) > at > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) > at java.net.URLClassLoader.access$100(URLClassLoader.java:71) > at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at > > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.marshallerSystemCache(IgnitionEx.java:2098) > at > > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.initializeDefaultCacheConfiguration(IgnitionEx.java:1914) > at > > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.initializeConfiguration(IgnitionEx.java:1899) > at > > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1573) > at > > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1547) > at > org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1003) > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:534) > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:515) > at org.apache.ignite.Ignition.start(Ignition.java:322) > at > org.apache.ignite.spark.IgniteContext.ignite(IgniteContext.scala:153) > at > org.apache.ignite.spark.IgniteContext.<init>(IgniteContext.scala:62) > at > training.PnLMergerVectorIgnite$.main(PnLMergerVectorIgnite.scala:50) > at training.PnLMergerVectorIgnite.main(PnLMergerVectorIgnite.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:672) > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) > at > org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > Caused by: java.lang.ClassNotFoundException: > javax.cache.configuration.MutableConfiguration > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > I know that the class javax.cache.configuration.MutableConfiguration > belongs > to cache-api-1.0.0.jar and I see it present in my jar-with-dependencies > > Thank you for looking at this. > > > > > -- > View this message in context: > http://apache-ignite-users.70518.x6.nabble.com/Ignite-Installation-with-Spark-under-CDH-tp4457p4507.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. >
