Thank you Aljoscha,
Sorry, but now the next problem occurs.
The code i've posted works fine locally (in eclipse). but in the cluster
environement there's a problem: NoClassDefFoundError: org/json4s/Formats
I'm not really sure wether this problem is because of stratosphere/yarn
or json4s.
A little bit weird is that the same functionality(and code) with same
dependencies in spark works fine in the cluster. without any error.
Getting the following error message:
eu.stratosphere.client.program.ProgramInvocationException: The program
execution failed: java.lang.NoClassDefFoundError: org/json4s/Formats
at TwitterWeather$$anon$17.map(TwitterWeather.scala:143)
at TwitterWeather$$anon$17.map(TwitterWeather.scala:143)
at
eu.stratosphere.pact.runtime.task.chaining.ChainedCollectorMapDriver.collect(ChainedCollectorMapDriver.java:71)
at TwitterWeather$$anon$16.map(TwitterWeather.scala:141)
at TwitterWeather$$anon$16.map(TwitterWeather.scala:141)
at
eu.stratosphere.pact.runtime.task.chaining.ChainedCollectorMapDriver.collect(ChainedCollectorMapDriver.java:71)
at
TwitterWeather$$anon$15$$anonfun$map$5.apply(TwitterWeather.scala:141)
at
TwitterWeather$$anon$15$$anonfun$map$5.apply(TwitterWeather.scala:141)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at TwitterWeather$$anon$15.map(TwitterWeather.scala:141)
at TwitterWeather$$anon$15.map(TwitterWeather.scala:141)
at
eu.stratosphere.pact.runtime.task.chaining.ChainedCollectorMapDriver.collect(ChainedCollectorMapDriver.java:71)
at
eu.stratosphere.pact.runtime.task.DataSourceTask.invoke(DataSourceTask.java:171)
at
eu.stratosphere.nephele.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:260)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.ClassNotFoundException: org.json4s.Formats
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 16 more
at eu.stratosphere.client.program.Client.run(Client.java:316)
at eu.stratosphere.client.program.Client.run(Client.java:282)
at eu.stratosphere.client.program.Client.run(Client.java:276)
at eu.stratosphere.client.program.Client.run(Client.java:220)
at
eu.stratosphere.client.CliFrontend.executeProgram(CliFrontend.java:327)
at eu.stratosphere.client.CliFrontend.run(CliFrontend.java:314)
at
eu.stratosphere.client.CliFrontend.parseParameters(CliFrontend.java:927)
at eu.stratosphere.client.CliFrontend.main(CliFrontend.java:951)
kind regards
Norman
