Hi Sean, There is no such operation in the streams DSL. This setup is fairly big-data specific which is outside the scope of Akka Streams. I imagine that there will be people writing such tools on top of Akka Streams though, but we have no plans to have this right now. Where Akka Streams can still enter the picture is to model the communication channel between the entities of the different layers, conveying backpressure to the senders -- but the shuffling logic is not covered.
-Endre On Mon, Jul 14, 2014 at 2:36 AM, Sean Zhong <[email protected]> wrote: > Hi Martynas, > > I am trying to do a data shuffle between two layers. > On the upstream, there are a layer with multiple actors. > On the downstream, there are another layer with multiple actors. > > Each upstream actor will distribute the message flowed by it to all > downstream actors, maybe groupped the message by a key hash function. And > the whole flow speed is controlled by backpressure. > > Is it possible to express this in Flow DSL? > > > On Friday, July 11, 2014 11:59:15 PM UTC+8, Martynas Mickevičius wrote: > >> Hi Sean, >> >> could you elaborate a bit mote what are you trying to do? >> >> >> On Thu, Jul 10, 2014 at 4:41 AM, Sean Zhong <[email protected]> wrote: >> >>> >>> >>> <https://lh3.googleusercontent.com/-CknzCjr2mNo/U73vEEZlsDI/AAAAAAAAD3Y/Sd-exFRr3aQ/s1600/akka_shuffle.png> >>> >>> >>> Take wordcount as an example, S is a list of files, >>> >>> Flow(S).foreach(fileName => >>> >>> val file = Source.fromFile(fileName) >>> >>> Flow(file.getLines()) >>> >>> .flatMap(splitWords) >>> >>> .groupBy(_) >>> >>> ... >>> >>> Ideas? >>> >>> >>> Sean >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ >>> current/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups.google.com/ >>> group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Martynas Mickevičius >> Typesafe <http://typesafe.com/> – Reactive >> <http://www.reactivemanifesto.org/> Apps on the JVM >> > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Akka Team Typesafe - The software stack for applications that scale Blog: letitcrash.com Twitter: @akkateam -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
