If possible, could you give us the root cause and solution for future readers of this thread.
On Wed, Nov 18, 2015 at 6:37 AM, swetha kasireddy <swethakasire...@gmail.com > wrote: > It works fine after some changes. > > -Thanks, > Swetha > > On Tue, Nov 17, 2015 at 10:22 PM, Tathagata Das <t...@databricks.com> > wrote: > >> Can you verify that the cluster is running the correct version of Spark. >> 1.5.2. >> >> On Tue, Nov 17, 2015 at 7:23 PM, swetha kasireddy < >> swethakasire...@gmail.com> wrote: >> >>> Sorry <scope>compile</scope> makes it work locally. But, the cluster >>> still seems to have issues with <scope>provided</scope>. Basically it >>> does not seem to process any records, no data is shown in any of the tabs >>> of the Streaming UI except the Streaming tab. Executors, Storage, Stages >>> etc show empty RDDs. >>> >>> On Tue, Nov 17, 2015 at 7:19 PM, swetha kasireddy < >>> swethakasire...@gmail.com> wrote: >>> >>>> Hi TD, >>>> >>>> Basically, I see two issues. With <scope>provided</scope> the job does >>>> not start localy. It does start in Cluster but seems no data is >>>> getting processed. >>>> >>>> Thanks, >>>> Swetha >>>> >>>> On Tue, Nov 17, 2015 at 7:04 PM, Tim Barthram <tim.barth...@iag.com.au> >>>> wrote: >>>> >>>>> If you are running a local context, could it be that you should use: >>>>> >>>>> >>>>> >>>>> <scope>provided</scope> >>>>> >>>>> >>>>> >>>>> ? >>>>> >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Tim >>>>> >>>>> >>>>> >>>>> *From:* swetha kasireddy [mailto:swethakasire...@gmail.com] >>>>> *Sent:* Wednesday, 18 November 2015 2:01 PM >>>>> *To:* Tathagata Das >>>>> *Cc:* user >>>>> *Subject:* Re: Streaming Job gives error after changing to version >>>>> 1.5.2 >>>>> >>>>> >>>>> >>>>> This error I see locally. >>>>> >>>>> >>>>> >>>>> On Tue, Nov 17, 2015 at 5:44 PM, Tathagata Das <t...@databricks.com> >>>>> wrote: >>>>> >>>>> Are you running 1.5.2-compiled jar on a Spark 1.5.2 cluster? >>>>> >>>>> >>>>> >>>>> On Tue, Nov 17, 2015 at 5:34 PM, swetha <swethakasire...@gmail.com> >>>>> wrote: >>>>> >>>>> >>>>> >>>>> Hi, >>>>> >>>>> I see java.lang.NoClassDefFoundError after changing the Streaming job >>>>> version to 1.5.2. Any idea as to why this is happening? Following are >>>>> my >>>>> dependencies and the error that I get. >>>>> >>>>> <dependency> >>>>> <groupId>org.apache.spark</groupId> >>>>> <artifactId>spark-core_2.10</artifactId> >>>>> <version>${sparkVersion}</version> >>>>> <scope>provided</scope> >>>>> </dependency> >>>>> >>>>> >>>>> <dependency> >>>>> <groupId>org.apache.spark</groupId> >>>>> <artifactId>spark-streaming_2.10</artifactId> >>>>> <version>${sparkVersion}</version> >>>>> <scope>provided</scope> >>>>> </dependency> >>>>> >>>>> >>>>> <dependency> >>>>> <groupId>org.apache.spark</groupId> >>>>> <artifactId>spark-sql_2.10</artifactId> >>>>> <version>${sparkVersion}</version> >>>>> <scope>provided</scope> >>>>> </dependency> >>>>> >>>>> >>>>> <dependency> >>>>> <groupId>org.apache.spark</groupId> >>>>> <artifactId>spark-hive_2.10</artifactId> >>>>> <version>${sparkVersion}</version> >>>>> <scope>provided</scope> >>>>> </dependency> >>>>> >>>>> >>>>> >>>>> <dependency> >>>>> <groupId>org.apache.spark</groupId> >>>>> <artifactId>spark-streaming-kafka_2.10</artifactId> >>>>> <version>${sparkVersion}</version> >>>>> </dependency> >>>>> >>>>> >>>>> Exception in thread "main" java.lang.NoClassDefFoundError: >>>>> org/apache/spark/streaming/StreamingContext >>>>> at java.lang.Class.getDeclaredMethods0(Native Method) >>>>> at java.lang.Class.privateGetDeclaredMethods(Class.java:2693) >>>>> at java.lang.Class.privateGetMethodRecursive(Class.java:3040) >>>>> at java.lang.Class.getMethod0(Class.java:3010) >>>>> at java.lang.Class.getMethod(Class.java:1776) >>>>> at >>>>> com.intellij.rt.execution.application.AppMain.main(AppMain.java:125) >>>>> Caused by: java.lang.ClassNotFoundException: >>>>> org.apache.spark.streaming.StreamingContext >>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:372) >>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:360) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>>>> at >>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>> >>>>> >>>>> >>>>> -- >>>>> View this message in context: >>>>> http://apache-spark-user-list.1001560.n3.nabble.com/Streaming-Job-gives-error-after-changing-to-version-1-5-2-tp25406.html >>>>> Sent from the Apache Spark User List mailing list archive at >>>>> Nabble.com. >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>>> For additional commands, e-mail: user-h...@spark.apache.org >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _____________________________________________________________________ >>>>> >>>>> The information transmitted in this message and its attachments (if >>>>> any) is intended >>>>> only for the person or entity to which it is addressed. >>>>> The message may contain confidential and/or privileged material. Any >>>>> review, >>>>> retransmission, dissemination or other use of, or taking of any action >>>>> in reliance >>>>> upon this information, by persons or entities other than the intended >>>>> recipient is >>>>> prohibited. >>>>> >>>>> If you have received this in error, please contact the sender and >>>>> delete this e-mail >>>>> and associated material from any computer. >>>>> >>>>> The intended recipient of this e-mail may only use, reproduce, >>>>> disclose or distribute >>>>> the information contained in this e-mail and any attached files, with >>>>> the permission >>>>> of the sender. >>>>> >>>>> This message has been scanned for viruses. >>>>> _____________________________________________________________________ >>>>> >>>> >>>> >>> >> >
