There is a new API called repartitionAndSortWithinPartitions() in master, it may help in this case, then you should do the `groupBy()` by yourself.
On Wed, Oct 8, 2014 at 4:03 PM, chinchu <chinchu....@gmail.com> wrote: > Sean, > > I am having a similar issue, but I have a lot of data for a group & I cannot > materialize the iterable into a List or Seq in memory. [I tried & it runs > into OOM]. is there any other way to do this ? > > I also tried a secondary-sort, with the key having the "group::time", but > the problem with that is the same group-name ends up in multiple partitions > & I am having to run sortByKey with one partition - sortByKey(true, 1) which > shuffles a lot of data.. > > Thanks, > -C > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/GroupBy-Key-and-then-sort-values-with-the-group-tp14455p15990.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
