Hi Patrik, I will do so when I have time to write up my case (on the RS generic level), in the next few days.
Thanks! Daniel On Wed, Apr 23, 2014 at 12:36 PM, Patrik Nordwall <[email protected] > wrote: > > > > On Tue, Apr 22, 2014 at 8:26 PM, Daniel Armak <[email protected]> wrote: > >> On Tue, Apr 22, 2014 at 1:42 PM, Patrik Nordwall < >> [email protected]> wrote: >> >>> Yes, uncompressing processor must be able to buffer and send smaller >>> chunks downstream. If it has a compression ratio of 10:1, and it gets >>> demand from downstream of 1000 chunks it should request 100 chunks from its >>> upstream. It can not know exactly, but it can do a fairly good guess, and >>> even adapt dynamically. >>> >> >> IIUC, you expect each processor to be explicitly configured to know the >> desired chunk size, for the entire stream or for each producer/consumer >> pair separately. Or at least to try to preserve the input chunk size in its >> output (which is not always the correct behavior). >> >> What worries me is the lack of a generic interface to handle this setup. >> If I get a Producer<ByteString> as a parameter, I have no way to >> communicate to it what size chunks I need; I have to trust that whoever >> created the actual producer implementation instance configured it >> correctly. I can't enforce the constraints that the reactive streams SPI is >> supposed to guarantee, such as not getting too much data as input. >> >> It's also bad for separation of mechanism (processors) and policy (chunk >> size). One day we'll have support for deploying an akka-stream on top of >> akka-remoting, across physical nodes. The stream (materializer?) will want >> to control chunk size and buffering policy, which should be different >> between local and remote links. Without a standard interface, the >> materializer wouldn't be able to tell a black-box Producer to give it >> chunks of size X. >> >> I expect projects will start creating local solutions to this problem (I >> think akka-http will need one), and I hope eventually one solution will >> become a standard library, but it surprises me that the Reactive Streams >> designers don't think this is important enough to be part of the API . (As >> an optional, but standardized, extra interface a Producer/Consumer can >> implement.) >> > > Thanks for your feedback. You're welcome to start a broader discussion at > the reactive-streams repo. > https://github.com/reactive-streams/reactive-streams/issues > > /Patrik > > >> >> Thanks, >> >> Daniel Armak >> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > > Patrik Nordwall > Typesafe <http://typesafe.com/> - Reactive apps on the JVM > Twitter: @patriknw > JOIN US. REGISTER TODAY! <http://www.scaladays.org/> > Scala <http://www.scaladays.org/> > Days <http://www.scaladays.org/> > June 16th-18th, <http://www.scaladays.org/> > Berlin <http://www.scaladays.org/> > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
