Re: Spark Streaming - how to implement multiple calculation using the same data set

2014-09-02 Thread Alireza Salemi
Tobias, That was what I was planing to do and technical lead is the opinion that we should some how process a message only once and calculate all the measures for the worker. I was wondering if there is a solution out there for that? Thanks, Ali > Hi, > > On Wed, Sep 3, 2014 at 6:54 AM, salemi

Re: Spark Streaming - how to implement multiple calculation using the same data set

2014-09-02 Thread Tobias Pfeiffer
Hi, On Wed, Sep 3, 2014 at 6:54 AM, salemi wrote: > I was able to calculate the individual measures separately and know I have > to merge them and spark streaming doesn't support outer join yet. > Can't you assign some dummy key (e.g., index) before your processing and then join on that key usi