Hi,
I have a use case that I believe Akka Streams may be well suited for, but
I'm not clearly seeing how to implement part of it from reading the
documentation.
I have a single, finite input source from an iterator stream. I want to use
Balance to split this stream into N partitions and then be able to feed
these stream partitions each into a RunnableFlow that would be executed on
its own node in a distributed system.
in ~> balance.out(0) ~> sink0
balance.out(1) ~> sink1
...
balance.out(n) ~> sinkN
I have some rough code sketched together, but I think I'm missing some
concepts to tie everything together. Runtime complains about the inlet for
the UniformInOutShape which I'm assuming is what I have in the inputGraph.
Any nudge in the right direction would be appreciated:
val iteratorBuilder: () => Iterator[T] = ???
val inputGraph: Graph[UniformFanOutShape[T,T], Unit] = FlowGraph.partial() {
implicit builder : FlowGraph.Builder =>
import FlowGraph.Implicits._
val stream: SourceShape[T] = builder.add(Source(iteratorBuilder))
val balance = builder.add(Balance[T](numPartitions))
stream ~> balance
balance
}
partitions = (0 until numPartitions).map(i => {
val sink : Sink[T, Future[Stream[T]]] = Sink.fold(Stream.empty[T]) { (v, e)
=> v.+:(e) }
val outFlow: RunnableFlow[Future[Stream[T]]] = FlowGraph.closed(sink) {
implicit builder : FlowGraph.Builder => sink =>
import FlowGraph.Implicits._
val balance = builder.add(inputGraph)
balance.out(i) ~> sink
}
new Partition(i, outFlow)
}).toArray
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
