I think you can write the SQL query and run it usin sqlContext. like
select *,row_number() over(partitin by assetid order by assetid) rn from t On Thu, Sep 1, 2016 at 3:16 PM, saurabh3d <saurabh.s.du...@oracle.com> wrote: > Hi All, > > As per SPARK-11001 <https://issues.apache.org/jira/browse/SPARK-11001> > , > Window functions should be supported by SQLContext. But when i try to run > > SQLContext sqlContext = new SQLContext(jsc); > WindowSpec w = Window.partitionBy("assetId").orderBy("assetId"); > DataFrame df_2 = df1.withColumn("row_number", row_number().over(w)); > df_2.show(false); > > it fails with: > Exception in thread "main" org.apache.spark.sql.AnalysisException: Could > not > resolve window function 'row_number'. Note that, using window functions > currently requires a HiveContext; > > This code runs fine with HiveContext. > Any idea what’s going on? Is this a known issue and is there a workaround > to make Window function work without HiveContext. > > Thanks, > Saurabh > > > > > -- > View this message in context: http://apache-spark-user-list. > 1001560.n3.nabble.com/Window-Functions-with-SQLContext-tp27636.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > > -- Best Regards, Ayan Guha
