[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179252#comment-14179252 ]
koert kuipers commented on SPARK-3655: -------------------------------------- hey matei, i was referring to the partition by one key and values sorted by another. does adding a keyOrdering in the ShuffleDependency lead to an efficient (secondary) sorting on that key in the sort-based shuffle? i haven't looked at the code yet for SortShuffleManager and SortShuffleWriter... > Secondary sort > -------------- > > Key: SPARK-3655 > URL: https://issues.apache.org/jira/browse/SPARK-3655 > Project: Spark > Issue Type: New Feature > Components: Spark Core > Affects Versions: 1.1.0 > Reporter: koert kuipers > Priority: Minor > > Now that spark has a sort based shuffle, can we expect a secondary sort soon? > There are some use cases where getting a sorted iterator of values per key is > helpful. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org