sorry typo, I meant *without* the addJar On 14 December 2015 at 11:13, Jakob Odersky <joder...@gmail.com> wrote:
> > Sorry,I'm late.I try again and again ,now I use spark 1.4.0 ,hadoop > 2.4.1.....but I also find something strange like this : > > > > http://apache-spark-user-list.1001560.n3.nabble.com/worker-java-lang-ClassNotFoundException-ttt-test-anonfun-1-td25696.html > > (if i use "textFile",It can't run.) > > In the link you sent, there is still an > `addJar(spark-assembly-hadoop-xx)`, can you try running your application > with that? > > On 11 December 2015 at 03:08, Bonsen <hengbohe...@126.com> wrote: > >> Thank you,and I find the problem is my package is test,but I write >> package org.apache.spark.examples ,and IDEA had imported the >> spark-examples-1.5.2-hadoop2.6.0.jar ,so I can run it,and it makes lots of >> problems.... >> ______________________________________________________________________ >> Now , I change the package like this: >> >> package test >> import org.apache.spark.SparkConf >> import org.apache.spark.SparkContext >> object test { >> def main(args: Array[String]) { >> val conf = new >> SparkConf().setAppName("mytest").setMaster("spark://Master:7077") >> val sc = new SparkContext(conf) >> sc.addJar("/home/hadoop/spark-assembly-1.5.2-hadoop2.6.0.jar")//It >> doesn't work.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!????? >> val rawData = sc.textFile("/home/hadoop/123.csv") >> val secondData = rawData.flatMap(_.split(",").toString) >> println(secondData.first) /////line 32 >> sc.stop() >> } >> } >> it causes that: >> 15/12/11 18:41:06 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, >> 219.216.65.129): java.lang.ClassNotFoundException: test.test$$anonfun$1 >> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> at java.lang.Class.forName0(Native Method) >> at java.lang.Class.forName(Class.java:348) >> at >> org.apache.spark.serializer.JavaDeserializationStream$$anon$1.resolveClass(JavaSerializer.scala:67) >> .... >> ________ >> // 219.216.65.129 is my worker computer. >> // I can connect to my worker computer. >> // Spark can start successfully. >> // addFile is also doesn't work,the tmp file will also dismiss. >> >> >> >> >> >> >> At 2015-12-10 22:32:21, "Himanshu Mehra [via Apache Spark User List]" >> <[hidden >> email] <http:///user/SendEmail.jtp?type=node&node=25689&i=0>> wrote: >> >> You are trying to print an array, but anyway it will print the objectID >> of the array if the input is same as you have shown here. Try flatMap() >> instead of map and check if the problem is same. >> >> --Himanshu >> >> ------------------------------ >> If you reply to this email, your message will be added to the discussion >> below: >> >> http://apache-spark-user-list.1001560.n3.nabble.com/HELP-I-get-java-lang-String-cannot-be-cast-to-java-lang-Intege-for-a-long-time-tp25666p25667.html >> To unsubscribe from HELP! I get "java.lang.String cannot be cast to >> java.lang.Intege " for a long time., click here. >> NAML >> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >> >> >> >> >> >> ------------------------------ >> View this message in context: Re:Re: HELP! I get "java.lang.String >> cannot be cast to java.lang.Intege " for a long time. >> <http://apache-spark-user-list.1001560.n3.nabble.com/HELP-I-get-java-lang-String-cannot-be-cast-to-java-lang-Intege-for-a-long-time-tp25666p25689.html> >> >> Sent from the Apache Spark User List mailing list archive >> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com. >> > >
