Thanks for all the support. It was a code issue which I overlooked it Regards, Satish Chandra
On Mon, Nov 23, 2015 at 3:49 PM, satish chandra j <jsatishchan...@gmail.com> wrote: > Sorry, just to understand my issue.....if Eclipse could not understand > Scala syntax properly than it should error for the other Spark Job which is > fetching data from a RDBM and printing the output in console. I think there > is some dependencies which are missing due to which "import > sqlContext.implicits._" is not recognized during compile time > > Please let me know if any further inputs needed to fix the same > > Regards, > Satish Chandra > > On Mon, Nov 23, 2015 at 3:29 PM, prosp4300 <prosp4...@163.com> wrote: > >> >> >> So it is actually a compile time error in Eclipse, instead of jar >> generation from Eclipse, you can try to use sbt to assembly your jar, looks >> like your Eclipse does not recognize the Scala syntax properly. >> >> >> >> At 2015-11-20 21:36:55, "satish chandra j" <jsatishchan...@gmail.com> >> wrote: >> >> HI All, >> I am getting this error while generating executable Jar file itself in >> Eclipse, if the Spark Application code has "import sqlContext.implicits._" >> line in it. Spark Applicaiton code works fine if the above mentioned line >> does not exist as I have tested by fetching data from an RDBMS by >> implementing JDBCRDD >> >> I tried couple of DataFrame related methods for which most of them errors >> stating that method has been overloaded >> >> Please let me know if any further inputs needed to analyze it >> >> Regards, >> Satish Chandra >> >> On Fri, Nov 20, 2015 at 5:46 PM, prosp4300 <prosp4...@163.com> wrote: >> >>> >>> Looks like a classpath problem, if you can provide the command you used >>> to run your application and environment variable SPARK_HOME, it will help >>> others to identify the root problem >>> >>> >>> 在2015年11月20日 18:59，Satish <jsatishchan...@gmail.com> 写道: >>> >>> Hi Michael, >>> As my current Spark version is 1.4.0 than why it error out as "error: >>> not found: value sqlContext" when I have "import sqlContext.implicits._" in >>> my Spark Job >>> >>> Regards >>> Satish Chandra >>> ------------------------------ >>> From: Michael Armbrust <mich...@databricks.com> >>> Sent: 20-11-2015 01:36 >>> To: satish chandra j <jsatishchan...@gmail.com> >>> Cc: user <user@spark.apache.org>; hari krishna <harikrishn...@gmail.com> >>> Subject: Re: Error not found value sqlContext >>> >>> >>> http://spark.apache.org/docs/latest/sql-programming-guide.html#upgrading-from-spark-sql-10-12-to-13 >>> >>> On Thu, Nov 19, 2015 at 4:19 AM, satish chandra j < >>> jsatishchan...@gmail.com> wrote: >>> >>>> HI All, >>>> we have recently migrated from Spark 1.2.1 to Spark 1.4.0, I am >>>> fetching data from an RDBMS using JDBCRDD and register it as temp table to >>>> perform SQL query >>>> >>>> Below approach is working fine in Spark 1.2.1: >>>> >>>> JDBCRDD --> apply map using Case Class --> apply createSchemaRDD --> >>>> registerTempTable --> perform SQL Query >>>> >>>> but now as createSchemaRDD is not supported in Spark 1.4.0 >>>> >>>> JDBCRDD --> apply map using Case Class with* .toDF()* --> >>>> registerTempTable --> perform SQL query on temptable >>>> >>>> >>>> JDBCRDD --> apply map using Case Class --> RDD*.toDF()*.registerTempTable >>>> --> perform SQL query on temptable >>>> >>>> Only solution I get everywhere is to use "import >>>> sqlContext.implicits._" after val SQLContext = new >>>> org.apache.spark.sql.SQLContext(sc) >>>> >>>> But it errors with the two generic errors >>>> >>>> *1. error: not found: value sqlContext* >>>> >>>> *2. value toDF is not a member of org.apache.spark.rdd.RDD* >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> >>> >> >> >> >> > >
