You might want to explore: a) check what class loader Spark passes to the ActorSystem b) setting the current Threads ContextClassLoader to be the appropriate one around the thing-block c) try to load the class Akka reports as missing from the ClassLoader you're using
(You have entered the world of class loaders, a world of advice: Here be dragons) On Fri, Apr 4, 2014 at 12:54 AM, Bryn Keller <[email protected]> wrote: > Hi Viktor, > > Sure, here's some more information, please let me know if there are other > things that would help. The code looks basically like this: > > object Boot extends App { > > val sparkLoader = { > val sparkJar = Directory.Current.get / "lib" / "spark-client.jar" > val loader = new URLClassLoader(Array(sparkJar.toURL), > getClass.getClassLoader) > loader > } > > val thing = { > val class_ = sparkLoader.loadClass("org.my.ClassName") > val instance = class_.newInstance() > instance.asInstanceOf[MyInterface] > } > } > > So Boot lives in one jar with all its dependencies, call it boot.jar. I > run it with java -jar boot.jar. In the lib folder of the current directory > is spark-client.jar, which includes org.my.Classname as well as Spark and > all its dependencies. Both boot.jar and spark-client.jar were prepared > using the sbt assembly plugin. org.my.ClassName starts a SparkContext in > its constructor, which is what leads to the Akka exception. > > Thanks, > Bryn > > > On Thu, Apr 3, 2014 at 3:30 PM, √iktor Ҡlang <[email protected]>wrote: > >> Hi Bryn, >> >> you aren't telling us how you are running this, so it becomes impossible >> (my crystal ball is always in for repairs) to divine why the classloader >> you supply to your ActorSystem creation does not have access to the >> akka-remote jar. >> >> Could you share that? >> >> >> On Fri, Apr 4, 2014 at 12:17 AM, Bryn Keller <[email protected]> wrote: >> >>> Hi Folks, >>> >>> I have two 3rd party libraries I need to use in my application. Both use >>> Akka, but they use different versions (2.1 and 2.3, I think). Since I need >>> to use these versions together in the same JVM (I don't need 2.1 and 2.3 to >>> talk to each other, just share a VM), I started looking at loading these >>> libraries using separate classloaders. I ran into this exception, which >>> (just from looking at the stack trace) makes me think that Akka is trying >>> to load classes in an unusual way. ReflectiveDynamicAccess is the name that >>> makes me wonder this. >>> >>> Whatever the reason, Akka can't find classes that are in the same jar >>> with the classes that had to be loaded in order to even get this far. Does >>> anyone have suggestions about better ways to deal with this situation? Is >>> anyone using OSGi for this kind of thing? Please let me know if the >>> question isn't clear or if I'm overlooking something simple. >>> >>> Thanks, >>> Bryn >>> >>> Exception in thread "main" java.lang.ClassNotFoundException: >>> akka.remote.RemoteActorRefProvider >>> at java.net.URLClassLoader$1.run(Unknown Source) >>> at java.net.URLClassLoader$1.run(Unknown Source) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(Unknown Source) >>> at java.lang.ClassLoader.loadClass(Unknown Source) >>> at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) >>> at java.lang.ClassLoader.loadClass(Unknown Source) >>> at java.lang.Class.forName0(Native Method) >>> at java.lang.Class.forName(Unknown Source) >>> at >>> akka.actor.ReflectiveDynamicAccess$$anonfun$getClassFor$1.apply(DynamicAccess.scala:67) >>> at >>> akka.actor.ReflectiveDynamicAccess$$anonfun$getClassFor$1.apply(DynamicAccess.scala:66) >>> at scala.util.Try$.apply(Try.scala:161) >>> at >>> akka.actor.ReflectiveDynamicAccess.getClassFor(DynamicAccess.scala:66) >>> at >>> akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:84) >>> at >>> akka.actor.ActorSystemImpl.liftedTree1$1(ActorSystem.scala:584) >>> at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:577) >>> at akka.actor.ActorSystem$.apply(ActorSystem.scala:141) >>> at akka.actor.ActorSystem$.apply(ActorSystem.scala:118) >>> at >>> org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:96) >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: >>> http://doc.akka.io/docs/akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: >>> https://groups.google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Cheers, >> √ >> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Cheers, √ -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
