Tobias, That was what I was planing to do and technical lead is the opinion that we should some how process a message only once and calculate all the measures for the worker.
I was wondering if there is a solution out there for that? Thanks, Ali > Hi, > > On Wed, Sep 3, 2014 at 6:54 AM, salemi <alireza.sal...@udo.edu> wrote: > >> I was able to calculate the individual measures separately and know I >> have >> to merge them and spark streaming doesn't support outer join yet. >> > > Can't you assign some dummy key (e.g., index) before your processing and > then join on that key using a function from > http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.streaming.dstream.PairDStreamFunctions > ? > > Tobias > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
