N Campbell created SPARK-10747:
----------------------------------
Summary: add support for window specification to include how NULLS
are ordered
Key: SPARK-10747
URL: https://issues.apache.org/jira/browse/SPARK-10747
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 1.5.0
Reporter: N Campbell
You cannot express how NULLS are to be sorted in the window order specification
and have to use a compensating expression to simulate.
Error: org.apache.spark.sql.AnalysisException: line 1:76 missing ) at 'nulls'
near 'nulls'
line 1:82 missing EOF at 'last' near 'nulls';
SQLState: null
Same limitation as Hive reported in Apache JIRA HIVE-9535 )
This fails
select rnum, c1, c2, c3, dense_rank() over(partition by c1 order by c3 desc
nulls last) from tolap
select rnum, c1, c2, c3, dense_rank() over(partition by c1 order by case when
c3 is null then 1 else 0 end) from tolap
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
