[
https://issues.apache.org/jira/browse/SPARK-11055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li closed SPARK-11055.
---------------------------
Resolution: Duplicate
> Use mixing hash-based and sort-based aggregation in
> TungstenAggregationIterator
> -------------------------------------------------------------------------------
>
> Key: SPARK-11055
> URL: https://issues.apache.org/jira/browse/SPARK-11055
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Liang-Chi Hsieh
>
> In TungstenAggregationIterator we switch to sort-based aggregation when we
> can't allocate more memory for hashmap.
> However, using external sorter-based aggregation will write too much
> key-value pairs into disk. We should use mixing hash-based and sort-based
> aggregation to reduce the key-value pairs needed to write to disk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
