Hi Divya,
Then, how can https://issues.apache.org/jira/browse/SPARK-11001 be resolved. Thanks, Saurabh From: Divya Gehlot [mailto:divya.htco...@gmail.com] Sent: 01 September 2016 11:33 To: saurabh3d Cc: user @spark Subject: Re: Window Functions with SQLContext Hi Saurabh, Even I am using Spark 1.6+ version ..and when I didnt create hiveContext it threw the same error . So have to create HiveContext to access windows function Thanks, Divya On 1 September 2016 at 13:16, saurabh3d <HYPERLINK "mailto:saurabh.s.du...@oracle.com"; \nsaurabh.s.du...@oracle.com> wrote: Hi All, As per SPARK-11001 <https://issues.apache.org/jira/browse/SPARK-11001> , Window functions should be supported by SQLContext. But when i try to run SQLContext sqlContext = new SQLContext(jsc); WindowSpec w = Window.partitionBy("assetId").orderBy("assetId"); DataFrame df_2 = df1.withColumn("row_number", row_number().over(w)); df_2.show(false); it fails with: Exception in thread "main" org.apache.spark.sql.AnalysisException: Could not resolve window function 'row_number'. Note that, using window functions currently requires a HiveContext; This code runs fine with HiveContext. Any idea what’s going on? Is this a known issue and is there a workaround to make Window function work without HiveContext. Thanks, Saurabh -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Window-Functions-with-SQLContext-tp27636.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe e-mail: HYPERLINK "mailto:user-unsubscr...@spark.apache.org"user-unsubscr...@spark.apache.org
