[
https://issues.apache.org/jira/browse/SPARK-32934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiaan.geng updated SPARK-32934:
-------------------------------
Description:
Spark SQL support some window function like NTH_VALUE
If we specify window frame like
{code:java}
UNBOUNDED PRECEDING AND CURRENT ROW
{code}
or
{code:java}
UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
{code}
We can elimate some calculations.
For example: if we execute the SQL show below:
{code:java}
SELECT NTH_VALUE(col,
2) OVER(ORDER BY rank UNBOUNDED PRECEDING
AND CURRENT ROW)
FROM tab;
{code}
The output for row number greater than 1, return the fixed value. otherwise,
return null. So we just calculate the value once and notice whether the row
number less than 2.
UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING is simpler.
was:
Spark SQL support some window function like NTH_VALUE,FIRST_VALUE and
LAST_VALUE
If we specify window frame like
{code:java}
UNBOUNDED PRECEDING AND CURRENT ROW
{code}
or
{code:java}
UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
{code}
We can elimate some calculations.
For example: if we execute the SQL show below:
{code:java}
SELECT NTH_VALUE(col,
2) OVER(ORDER BY rank UNBOUNDED PRECEDING
AND CURRENT ROW)
FROM tab;
{code}
The output for row number greater than 1, return the fixed value. otherwise,
return null. So we just calculate the value once and notice whether the row
number less than 2.
UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING is simpler.
> Improve the performance for NTH_VALUE and Reactor the OffsetWindowFunction
> --------------------------------------------------------------------------
>
> Key: SPARK-32934
> URL: https://issues.apache.org/jira/browse/SPARK-32934
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.1.0
> Reporter: jiaan.geng
> Priority: Major
>
> Spark SQL support some window function like NTH_VALUE
> If we specify window frame like
> {code:java}
> UNBOUNDED PRECEDING AND CURRENT ROW
> {code}
> or
> {code:java}
> UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
> {code}
> We can elimate some calculations.
> For example: if we execute the SQL show below:
> {code:java}
> SELECT NTH_VALUE(col,
> 2) OVER(ORDER BY rank UNBOUNDED PRECEDING
> AND CURRENT ROW)
> FROM tab;
> {code}
> The output for row number greater than 1, return the fixed value. otherwise,
> return null. So we just calculate the value once and notice whether the row
> number less than 2.
> UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING is simpler.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
