Hi Sam, > 21 apr 2014 kl. 12:28 skrev Sam Halliday <sam.halli...@gmail.com>: > > Hi all, > > I am very excited by akka streams -- it aims to solve a problem that I see > time and time again. Every time I post to this list it feels like the > solution is always "wait until Akka Streams is released...". Finally, it is > here!
Yeah, exciting times. > > I intend to read the documentation fully, but I was a little disappointed > that the activator examples did not have a simple example with an > (effectively) infinite data source that can only be polled in serial, with > parallel (but controllably finite) consumers. Isn't that demonstrated with the random number generator source, and its slow consumers? > > Is there any chance of an example along these lines? > > A month or so ago, I asked the same of the RxJava community and it turned out > that it was a work-in-progess... so I created this little example comparing > various approaches (I didn't write an Akka Actor implementation because it is > quite obvious that it would just OOM): > > > https://github.com/fommil/rx-playground/blob/master/src/main/scala/com/github/fommil/rx/scratch.scala > > The `ProducerObservableParser` reads in a CSV file one line at a time (the > file is far too big to hold in memory), and then processes N rows in > parallel, only reading more lines as the consumers finish each row. There is > never more than a bounded number of rows in memory at any given point in time. That sounds very doable with akka streams. You can control the buffer sizes with the settings of the materializer. A consumer always signals upstream how many more elements it can handle, and the producer is not allowed to send more elements downstream than what was requested. > > The RxJava POC Observable is here > > > https://github.com/fommil/rx-playground/blob/master/src/main/scala/com/github/fommil/rx/producers.scala > > > But what is the equivalent Akka Streams code? The BasicTransformation example > reads in the whole text before "flowing" it, and I couldn't see anything > where the consuming was happening in parallel. BasicTransformation defines the input text in code (to make it simple), but the iterator next() is not called more than what can be consumed downstream. Isn't the log file sample more similar to your text file input? It does not read the whole file (if it was large) into memory. /Patrik > > Best regards, > Sam > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
