[
https://issues.apache.org/jira/browse/SPARK-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matei Zaharia updated SPARK-2047:
---------------------------------
Priority: Major (was: Minor)
> Use less memory in AppendOnlyMap.destructiveSortedIterator
> ----------------------------------------------------------
>
> Key: SPARK-2047
> URL: https://issues.apache.org/jira/browse/SPARK-2047
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Matei Zaharia
>
> This method tries to sort an the key-value pairs in the map in-place but ends
> up allocating a Tuple2 object for each one, which allocates a nontrivial
> amount of memory (32 or more bytes per entry on a 64-bit JVM). We could
> instead try to sort the objects in-place within the "data" array, or allocate
> an int array with the indices and sort those using a custom comparator. The
> latter is probably easiest to begin with.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
