[
https://issues.apache.org/jira/browse/SPARK-32934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiaan.geng updated SPARK-32934:
-------------------------------
Summary: Improve the performance for NTH_VALUE and Reactor the
OffsetWindowFunction (was: Improve the performance for
NTH_VALUE,FIRST_VALUE,LAST_VALUE)
> Improve the performance for NTH_VALUE and Reactor the OffsetWindowFunction
> --------------------------------------------------------------------------
>
> Key: SPARK-32934
> URL: https://issues.apache.org/jira/browse/SPARK-32934
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.1.0
> Reporter: jiaan.geng
> Priority: Major
>
> Spark SQL support some window function like NTH_VALUE,FIRST_VALUE and
> LAST_VALUE
> If we specify window frame like
> {code:java}
> UNBOUNDED PRECEDING AND CURRENT ROW
> {code}
> or
> {code:java}
> UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
> {code}
> We can elimate some calculations.
> For example: if we execute the SQL show below:
> {code:java}
> SELECT NTH_VALUE(col,
> 2) OVER(ORDER BY rank UNBOUNDED PRECEDING
> AND CURRENT ROW)
> FROM tab;
> {code}
> The output for row number greater than 1, return the fixed value. otherwise,
> return null. So we just calculate the value once and notice whether the row
> number less than 2.
> UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING is simpler.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
