Re: Dataset reduceByKey

2016-05-26 Thread Reynold Xin
Here's a ticket: https://issues.apache.org/jira/browse/SPARK-15598 On Fri, May 20, 2016 at 12:35 AM, Reynold Xin wrote: > Andres - this is great feedback. Let me think about it a little bit more > and reply later. > > > On Thu, May 19, 2016 at 11:12 AM, Andres Perez

Re: Dataset reduceByKey

2016-05-20 Thread Reynold Xin
Andres - this is great feedback. Let me think about it a little bit more and reply later. On Thu, May 19, 2016 at 11:12 AM, Andres Perez wrote: > Hi all, > > We were in the process of porting an RDD program to one which uses > Datasets. Most things were easy to transition,

Dataset reduceByKey

2016-05-19 Thread Andres Perez
Hi all, We were in the process of porting an RDD program to one which uses Datasets. Most things were easy to transition, but one hole in functionality we found was the ability to reduce a Dataset by key, something akin to PairRDDFunctions.reduceByKey. Our first attempt of adding the