Add one more library libraryDependencies += "org.apache.spark" % "spark-hive_2.10" % "1.2.0"
val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc) repalce sqlContext with hiveContext. Its working while using HiveContext for me. Devan M.S. | Research Associate | Cyber Security | AMRITA VISHWA VIDYAPEETHAM | Amritapuri | Cell +919946535290 | On Tue, Jan 20, 2015 at 4:45 PM, DEVAN M.S. <msdeva...@gmail.com> wrote: > Which context are you using HiveContext or SQLContext ? Can you try with > HiveContext > ?? > > > Devan M.S. | Research Associate | Cyber Security | AMRITA VISHWA > VIDYAPEETHAM | Amritapuri | Cell +919946535290 | > > > On Tue, Jan 20, 2015 at 3:49 PM, Xuelin Cao <xuelincao2...@gmail.com> > wrote: > >> >> Hi, I'm using Spark 1.2 >> >> >> On Tue, Jan 20, 2015 at 5:59 PM, Wang, Daoyuan <daoyuan.w...@intel.com> >> wrote: >> >>> Hi Xuelin, >>> >>> >>> >>> What version of Spark are you using? >>> >>> >>> >>> Thanks, >>> >>> Daoyuan >>> >>> >>> >>> *From:* Xuelin Cao [mailto:xuelincao2...@gmail.com] >>> *Sent:* Tuesday, January 20, 2015 5:22 PM >>> *To:* User >>> *Subject:* IF statement doesn't work in Spark-SQL? >>> >>> >>> >>> >>> >>> Hi, >>> >>> >>> >>> I'm trying to migrate some hive scripts to Spark-SQL. However, I >>> found some statement is incompatible in Spark-sql. >>> >>> >>> >>> Here is my SQL. And the same SQL works fine in HIVE environment. >>> >>> >>> >>> SELECT >>> >>> *if(ad_user_id>1000, 1000, ad_user_id) as user_id* >>> >>> FROM >>> >>> ad_search_keywords >>> >>> >>> >>> What I found is, the parser reports error on the "*if*" statement: >>> >>> >>> >>> No function to evaluate expression. type: AttributeReference, tree: >>> ad_user_id#4 >>> >>> >>> >>> >>> >>> Anyone have any idea about this? >>> >>> >>> >>> >>> >> >> >
