[
https://issues.apache.org/jira/browse/SPARK-53885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-53885:
----------------------------------
Issue Type: Umbrella (was: New Feature)
> Frequency estimation functions
> ------------------------------
>
> Key: SPARK-53885
> URL: https://issues.apache.org/jira/browse/SPARK-53885
> Project: Spark
> Issue Type: Umbrella
> Components: SQL
> Affects Versions: 4.1.0
> Reporter: Gengliang Wang
> Assignee: Yuchuan Huang
> Priority: Major
> Fix For: 4.1.0
>
>
> Introduce the following frequency estimation functions:
> * *APPROX_TOP_K(expr[, k[, maxItemsTracked]])* – Returns an approximate list
> of the top _k_ most frequent values in the input expression using a
> probabilistic algorithm.
> * *APPROX_TOP_K_ACCUMULATE(expr[, maxItemsTracked])* – Creates a state
> object that accumulates frequency statistics for use in later estimation or
> combination.
> * *APPROX_TOP_K_ESTIMATE(state [, k] ])* – Extracts and returns the
> approximate top _k_ values and their estimated frequencies from an
> accumulated state.
> * *APPROX_TOP_K_COMBINE(expr[, maxItemsTracked])* – Merges intermediate
> APPROX_TOP_K state objects, allowing distributed or parallel computation.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
