> > > Now, with auto-fusing that changes. By default, if there's a split, the > branches won't be executed in parallel. I think that's quite a big > semantical change. > > > This is a bit of an overstatement: the semantics of the stream processing > do not change, the same rules apply to how elements flow and how > back-pressure is handled, all branches of a broadcast will make progress > concurrently—but not necessarily in parallel. This means that the > management of resource allocation has changed, not the semantics. >
Right, my wording was bad, though what the end-user observes could be quite different. I couldn't find the right words, but changing the default stream behaviour from "parallel" to "concurrent" describes it correctly I guess? (which then, at some level, can be viewed as different semantics ;) ) Probably also the fact that in my oversimplified tests I use Thread.sleep for simulating "work" plays a role ;) > A similar effect is when there are e.g. two computationally expensive, > consecutive .map stages (they will now be processed sequentially, not > concurrently), however with splits I think it was a very natural > expectation that things will be processed in parallel. > > > Not necessarily. This is why the user gets to decide by declaring (some > of) the branches as asynchronous. > Sure, though that's how I thought about broadcast, or merge. Maybe it's a common misconception, but probably it's just me :) > Hence I have four questions: > > 1. if I have a split, where should the async boundary go? My first attempt > was adding the async boundary attribute to the split graph stage itself, > but that didn't work. By experimentation, I have to add it to the stages > that are connected to the split's outputs. > > > I realize that in the final release frenzy we forgot to add documentation > on how this actually works, I apologize. In short, adding attributes > applies to the whole graph on which they are added, and adding an > asynchronous boundary adds that boundary around the graph that has > previously been constructed. This means that the right place is around the > flow that gets attached to the split’s outputs. Currently that looks like > > someFlow.via(Flow[...].<your ops > here>.withAttributes(Attributes.asyncBoundary)) > > which will run <your ops here> in one extra actor. > so if I have a split component, and I want branches to be run in parallel (different actors), I need to do: val branch1 = someFlow.withAttributes(asyncBoundary) val branch2 = someFlow.withAttributes(asyncBoundary) split.out1 ~> branch1 split.out2 ~> branch2 Correct? For linear flows, e.g.: Source(List(1, 2, 3)) .map(_ + 1) .withAttributes(asyncBoundary) .map(_ * 2) .to(Sink.ignore) here two actors will be created? First wrapping the Source & map(_ + 1), second the map(_ * 2) & sink? > > 2. how to insert async boundaries in linear pipelines? E.g.: > > Source(List(1, 2, 3)) > .map(_ + 1) > .addAttributes(Attributes.asyncBoundary) > > > This is an oversight, we’ll need to provide appropriate overrides of this > method in all subclasses (ticket > <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fakka%2Fakka%2Fissues%2F19261&sa=D&sntz=1&usg=AFQjCNEfY1IhROusDQU4j0f_TYmsIpCA_Q> > created). > Thx :) > 3. are you sure it's a good idea to make all processing stages fuseable by > default? E.g. conflate seems to be naturally concurrent, as it deals with > slower/faster components, which doesn't make much sense if everything is in > one thread :) > > > This is a misconception: conflate is not concurrent, it is aware of > back-pressure, and it works even better when fused because that removes > additional (implicit) buffers that can lead to rather surprising behavior. > All operations should be fusable, but I agree that the fusing algorithm > should become more intelligent in selecting which parts of a graph to > fuse—right now it is called “Fusing.aggressive” because it will fuse > everything it can. > True about the buffers, took me a while to understand one example ;) Having conflate only makes sense if there's some non-fused component out there I guess, but you are correct of course that it can be fused with things before/after it. > 4. which built-in stages are fuseable by default? The docs are quite vague > here saying only "linear" ones. Which ones are these? > > > It is easier to name the exceptions: SslTlsStage, groupBy, and some > sources and sinks (but for those the difference is typically not that > large). > Ah, ok, well I still think it might be a good idea to explicitly mark them as non-fuseable at least in the docs. Once you have to think about where to put the async boundaries, such information might be useful. Thanks for the answers! :) Adam -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
