If you are building the jar file using maven, you can also use the maven assembly plugin to build a fat jar (jar-with-dependencies). Then, the dependencies will be packed into the job's jar file.
On Mon, Aug 11, 2014 at 7:38 AM, Aljoscha Krettek <[email protected]> wrote: > Hi, > it seems you have to put the json4s jar into the lib folder of your Flink > (Stratosphere) installation on every Slave Node. Are you using yarn or our > own cluster management? > > Aljoscha > > > On Sun, Aug 10, 2014 at 10:36 PM, Norman Spangenberg < > [email protected]> wrote: > > > Thank you Aljoscha, > > Sorry, but now the next problem occurs. > > The code i've posted works fine locally (in eclipse). but in the cluster > > environement there's a problem: NoClassDefFoundError: org/json4s/Formats > > I'm not really sure wether this problem is because of stratosphere/yarn > > or json4s. > > A little bit weird is that the same functionality(and code) with same > > dependencies in spark works fine in the cluster. without any error. > > Getting the following error message: > > > > eu.stratosphere.client.program.ProgramInvocationException: The program > > execution failed: java.lang.NoClassDefFoundError: org/json4s/Formats > > at TwitterWeather$$anon$17.map(TwitterWeather.scala:143) > > at TwitterWeather$$anon$17.map(TwitterWeather.scala:143) > > at eu.stratosphere.pact.runtime.task.chaining. > > ChainedCollectorMapDriver.collect(ChainedCollectorMapDriver.java:71) > > at TwitterWeather$$anon$16.map(TwitterWeather.scala:141) > > at TwitterWeather$$anon$16.map(TwitterWeather.scala:141) > > at eu.stratosphere.pact.runtime.task.chaining. > > ChainedCollectorMapDriver.collect(ChainedCollectorMapDriver.java:71) > > at TwitterWeather$$anon$15$$anonfun$map$5.apply( > > TwitterWeather.scala:141) > > at TwitterWeather$$anon$15$$anonfun$map$5.apply( > > TwitterWeather.scala:141) > > at scala.collection.Iterator$class.foreach(Iterator.scala:727) > > at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) > > at TwitterWeather$$anon$15.map(TwitterWeather.scala:141) > > at TwitterWeather$$anon$15.map(TwitterWeather.scala:141) > > at eu.stratosphere.pact.runtime.task.chaining. > > ChainedCollectorMapDriver.collect(ChainedCollectorMapDriver.java:71) > > at eu.stratosphere.pact.runtime.task.DataSourceTask.invoke( > > DataSourceTask.java:171) > > at eu.stratosphere.nephele.execution.RuntimeEnvironment. > > run(RuntimeEnvironment.java:260) > > at java.lang.Thread.run(Thread.java:744) > > Caused by: java.lang.ClassNotFoundException: org.json4s.Formats > > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > > at java.security.AccessController.doPrivileged(Native Method) > > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > > ... 16 more > > > > at eu.stratosphere.client.program.Client.run(Client.java:316) > > at eu.stratosphere.client.program.Client.run(Client.java:282) > > at eu.stratosphere.client.program.Client.run(Client.java:276) > > at eu.stratosphere.client.program.Client.run(Client.java:220) > > at eu.stratosphere.client.CliFrontend.executeProgram( > > CliFrontend.java:327) > > at eu.stratosphere.client.CliFrontend.run(CliFrontend.java:314) > > at eu.stratosphere.client.CliFrontend.parseParameters( > > CliFrontend.java:927) > > at eu.stratosphere.client.CliFrontend.main(CliFrontend.java:951) > > > > kind regards > > Norman > > > > >
